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I 


FOREWORD 


The  U.S.  Navy,  facing  rising  costs  associated  with  training,  has 
attempted  to  improve  cost-effectiveness  relative  to  training  by  devel- 
oping a self-paced  Computer  Managed  Instruction  (CMI)  System.  Initial 
CMI  training  systems  have  been  successful.  Further  methods  for  devel- 
oping even  greater  savings  have  been  examined  including  the  use  of  com- 
munication satellites  and  other  technologies  as  a delivery  vehicle  for 
CMI  at  remote  operational  sites.  The  project  title  was  originally  de- 
rived from  the  Communication  Satellite  Concept  and  is  called  Computer 
Managed  Instruction  by  Satellite  (COMISAT) . 

The  project  is  sponsored  by  the  Cybernetics  Technology  Office, 
Defense  Advanced  Research  Projects  Agency  (ARPA)  and  the  Research  and 
Program  Development  Office,  Chief  of  Naval  Education  and  Training 
(CNET) . CNET's  Training  Analysis  and  Evaluation  Group  (TAEG)  serves  as 
the  COMISAT  Project  Officer  for  CNET  and  as  the  Contracting  Officer's 
Technical  Representative  (COTR) . 

Planning  Research  Corporation  Information  Sciences  Company,  work- 
ing with  ARPA  and  CNET/TAEG,  was  responsible  for  the  project  background 
research  and  has  responsibility  for  the  design  and  preparation  phases. 

Others  involved  in  the  project  include  various  commands  and  agen- 
cies of  the  U.S.  Navy.  OOMNAVTELOOMM  is  to  provide  the  facility  for 
the  demonstration?  NAVCOMMSTA  Stockton,  CA  is  to  provide  the  required 
students  for  the  demonstration;  the  Naval  Telecommunications  Division, 
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Chief  of  Naval  Operations,  is  to  approve  the  use  of  the  Navy  telecom- 
munications system  for  the  demonstration;  the  Management  Instructional 
Information  Systems  Activity  is  to  provide  computer  support;  the  Chief 
of  Naval  Technical  Training  is  to  provide  the  CMI  course  and  associated 
materials;  and  the  Navy  Personnel  Research  and  Development  Center  is  to 
provide  additional  assistance  in  those  areas  where  they  have  partici- 
pated in  relevant  research  activities. 
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Chapter  I 
INTRODUCTION 


A.  Background 

This  report  addresses  the  second  and  a portion  of  the  third  phase 
of  the  project  originally  entitled  Computer  Managed  Instruction  by 
Satellite  (COMISAT) . The  project  is  sponsored  by  the  Cybernetics 
Technology  Office,  Defense  Advanced  Research  Projects  Agency  (ARPA)  and 
the  Research  and  Program  Development  Office,  Chief  of  Naval  Education 
and  Training  (CNET) . CNET's  Training  Analysis  and  Evaluation  Group 
(TAEG)  serves  as  the  COMISAT  Project  Officer  for  CNET  and  as  the  Con- 
tracting Officer's  Technical  Representative  (COTR) . Planning  Research 
Corporation  Information  Sciences  Company  (PRC/ISC) , working  with  ARPA 
and  CNET/TAEG,  is  responsible  for  the  project  background  research  and 
the  design,  development,  implementation,  and  evaluation  of  the 
demonstration. 

1 . Project  Genesis 

The  acceleration  of  military  personnel  costs  has  caused  mili- 
tary personnel  managers  to  pursue  ways  of  maximizing  the  return  on  dol- 
lars expended  and,  where  possible,  to  stabilize  or  reduce  costs.  One 
major  cause  of  accelerated  costs  is  training,  a continuing  and  neces- 
sary requirement. 

In  an  attempt  to  address  the  training  cost  problem,  the  U.S. 
Navy  developed  a self-paced,  computer  managed  instruction  (CMI) 
system.  The  systems  approach  to  instructional  development  was  used  to 
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provide  a set  of  prioritized  skills  derived  frcn  task  analysis.  These 
skills  were  translated  into  learning  objectives  and  then  into  learning 
modules  with  accompanying  self-paced  learning  materials  and  performance 
measures.  The  learning  modules  were  then  automated  through  CMI. 

Thus  far,  the  achievements  of  the  CMI  system,  which  provides 
a means  for  guiding  and  counseling  students  through  a continuum  of  in- 
struction with  only  minimal  staff  support,  have  been  dramatic.  The 
system  has  significantly  reduced  course  time,  instructional  and  support 
personnel,  and  student  attrition;  it  has  significantly  Increased  stu- 
dent end-of-course  achievement  levels;  and  It  has  been  estimated  to 
have  saved  over  $10  million  in  py  1975  alone  (Reference  1). 

The  success  of  the  CMI  system  in  the  continental  United 
States  prompted  CNET  and  ARPA  to  become  interested  in  extending  the 
system  to  Navy  personnel  at  sea  or  other  remote  locations.  More  spe- 
cifically, the  question  is  being  asked:  Can  further  improvements  in 
resource  use  be  effected  by  delivering  CMI  supported  training  to  job 
sites? 


2.  Purpose  and  Objectives 

The  purpose  of  the  research  project  is  to  evaluate  the  cost- 
effectiveness  of  instructional  support  delivered  at  job  sites  under  the 
direction  and  control  of  a centralized  CMI  system.  The  specific  objec 
tives  of  the  effort  are  to  determine; 

• Whether  CMI  delivered  to  remote  sites  produces  the  same 
learning  effectiveness  as  CMI  does  in  the  learning  center 
envi ronment 

• Whether  the  attitudes  of  students,  trainers,  and  key  remote 
site  personnel  are  supportive  of  CMI  delivered  to  remote  sites 

• Whether  CMI  delivered  to  remote  sites  is  as  economical  as  CMI 
in  the  learning  center  environment 
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• Personnel  requirements 

• Personnel  training  requirements 

• Organization  and  management  structure  requirements 

• Remote  site  space  requirements  and  operational  procedures  for 
effective  use  of  a CMI  training  support  system 

• Equipment,  maintenance,  spare  parts,  and  logistics 
requirements 


3.  Project  Phases 

The  research  effort  has  five  phases:  a feasibility  study;  a 


demonstration  design;  the  demonstration  preparation;  the  demonstration; 
and  the  evaluation  of  the  demonstration. 


The  first  phase  included  gathering  background  data  and  infor- 
mation, establishing  resource  requirements,  and  setting  the  parameters 
for  the  demonstration;  this  was  the  subject  of  the  December  1976  TAEG 
Report  No.  44  (Reference  2).  The  second  phase,  the  primary  subject  of 
this  report,  involved  determining  the  most  useful  approach  to  conduct 
the  demonstration.  The  research  approach  to  be  undertaken  was  defined 
and  a detailed  plan  for  its  preparation,  execution,  and  evaluation  spe- 
cified. The  third  phase,  a small  portion  of  which  is  addressed  in  this 
report,  involves  preparation  activities;  and,  the  fourth  phase  includes 
the  execution  and  monitoring  of  the  planned  demonstration  activities. 

In  the  fourth  phase,  adjustments  or  changes  will  be  made  as  needed, 
data  and  information  collected  and  prepared  for  analysis,  and  initial 
analyses  conducted.  The  fifth  phase  involves  summing  up  the  results  of 
the  demonstration,  drawing  conclusions,  and  making  recommendations. 

It  should  be  noted  that  in  order  to  insure  that  the  develop- 
ment and  execution  of  the  demonstration  is  progressing  satisfactorily. 
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a formative  evaluation  is  being  conducted.  The  purpose  is  to  furnish 
project  personnel  with  information  helpful  to  them  in  assessing  the 
progress  of  the  demonstration. 

B.  Evolution  of  the  Design 

1.  Introduction 

The  fundamental  research  question  posed  at  the  outset  of  the 
CXIMISAT  project  was:  Can  training  costs  be  reduced  and  the  manning 
situation  of  remote  job  sites,  sea  or  land,  be  improved  by  providing 
CMI  training  onsite?  Two  approaches  for  implementing  CMI  operationally 
were  explored  during  the  feasibility  study  to  answer  this  question. 

The  highest  pay-off  potential  was  projected  to  involve  per- 
sonnel just  entering  the  service  who  would  be  partially  trained  at  an 
A-School  and  then  be  assigned  to  a remote  sea  or  land  site  for  there 
mainder  of  their  A-School  training.  The  rationale  was  that: 

• The  courses  on  CMI  were  primarily  suited  for  new  personnel 
because  they  were  A-School  courses  or  preparation  for 
A-School  courses 

• Reducing  time  at  training  centers  would  reduce  associated 
costs  while  improving  the  manning  situation,  particularly  on 
ships 

• Operational  readiness  possibly  could  be  improved  by  marrying 
the  formal  CMI  training  with  on-the-job  training  (OJT) 

Sea  sites  were  given  priority  because  they  were  thought  to  be 
a "worst  case"  environment  when  considering  operational  requirements 
and  their  limited  physical  facilities.  It  was  felt  that  if  it  worked 
in  this  environment,  CMI  supported  training  may  work  at  any  job  site. 

The  second  approach  was  to  provide  Navy  personnel  already  as- 
signed to  a job  site  with  an  existing  CMI  course  that  would  meet  a site 
training  need.  The  rationale  was  that; 
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• Reducing  the  need  for  formal  training  would  reduce  associated 
costs 

• Reducing  the  absence  of  personnel  due  to  formal  training 
would  improve  the  manning  situation 

e Operational  training  possibly  could  be  improved  by  providing 
and  combining  formal  training  with  normal  daily  job  duties 

In  both  approaches,  five  basic  requirements  had  to  be  met  to 
execute  a demonstration: 


a Approval  for  the  use  of  an  existing  communications  system 

e Approval  for  the  use  of  the  U.S.  Navy  CMI  computer  facility 

at  Millington,  Tennessee 

a Selection  of  a validated  Oil  course 

a Identification  and  commitment  of  an  operational  site  for  the 
conduct  of  the  demonstration 


a 


Identification  and  commitment  of  specific  U.S.  Navy  personnel 
to  participate  as  subjects  and  to  act  in  a support  capacity 


In  terms  of  the  preferred  approach,  the  first  four  requirements  were 
met.  Tentative  approval  was  obtained  to  use; 

a The  standard  Navy  communications  system  to  include  the  Navy 
satellite  system,  if  required 

a The  Navy  CMI  facility 

a The  Radioman  (RM)  CMI  course 

a NAVCOMMSTA  Stockton  (a  ship  could  not  be  obtained) 

Nevertheless,  BUPERS  did  not  grant  permission  for  the  personnel.  CNET 
had  requested  that  approximately  30  new  Navy  personnel  be  assigned  to 
the  experimental  group  to  be  partially  trained  at  the  A-School  and  then 
assigned  to  the  demonstration  site  as  additions  to  the  normal  comple- 
ment of  personnel. 
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This  was  to  insure  that  the  demonstration  site  would  not  be  penalized 
by  partially  trained  and  inexperienced  personnel  being  assigned  to  reg- 
ular billets.  SUPERS  maintained  that  by  assigning  additional  person- 
nel in  a single  rate  to  a single  site  the  billeting  limit  would  be 
exceeded  and,  at  the  same  time,  deprive  other  sites  of  personnel.  This 
is  particularly  critical  in  the  RM  field,  where  there  is  a shortage  of 
personnel. 

As  a result  of  the  SUPERS  decision,  the  second  research  approach 
was  pursued  and  approval  was  obtained  to  use: 

• The  standard  Navy  terrestrial  communications  system  (Because 
Stockton  was  accessible  with  good  quality  landline  the  need 
for  the  satellite  was  eliminated.) 

• The  Navy  CMI  facility 

• The  A-School  Basic  Electricity  and  Electronics  (BE/E)  CMI 
course 

• NAVCOWiSTA  Stockton 

• Existing  RM  personnel  at  Stockton 

2,  Stockton  Demonstration  Site 

NAVCOMMSTA  Stockton,  like  the  majority  of  the  operational 
sites,  has  a great  number  of  training  requirements,  but  a limited  bud- 
get to  meet  these  needs.  Many  of  the  courses  are  offered  at  locations 
a considerable  distance  from  Stockton,  thus  requiring  absence  from  the 
site  of  needed  personnel  and  the  expenditure  of  large  sums  of  money 
each  year  for  travel  and  per  diem.  The  Stockton  commander  sees  the 
COMISAT  project  concept  as  a possible  approach  for  meeting  some  of  the 
Stockton  training  needs. 

Since  there  are  a limited  number  of  U.S.  Navy  courses  sup- 
ported by  CMI  (all  related  to  A-School  training),  it  was  necessary  to 
find  a match  between  Stockton  training  needs  and  a CXI  course. 
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At  the  s^une  time  the  selected  course  must  permit  the  fulfillment  of  the 
project  purpose. 

3.  Demonstration  Course 

NAVCOMMSTA  Stockton  RM  personnel  have  a special  training  re- 
quirement which  provides  an  opportunity  to  test  an  existing  CMI 
course.  RM's  are  not  required  to  take  the  A-School  BE/E  course.  How- 
ever, as  time  goes  on  they  find  that  a BE/E  background  can  be  useful 
both  on  the  job,  and  in  taking  the  electricity  and  electronics  portion 
of  the  E4,  E5,  and  E6  rate  advancement  exams.  Exam  scores  of  Stockton 
personnel  provide  strong  evidence  of  the  need  for  BE/E  training. 

The  method  currently  being  used  to  acquire  a background  in 
electronics  and  electricity  is  to  enroll  in  a correspondence  course. 
However,  this  has  not  been  a popular  approach  due  to  the  difficulty  of 
the  materials  and  the  self  discipline  required.  An  attractive  alterna- 
tive is  to  use  the  first  14  modules  of  the  self-paced  CMI  supported 
BE/E  course  as  the  basis  for  study.  To  indicate  the  potential  interest 
in  such  an  offering,  the  Education  Service  Officer  (ESO)  at  Stockton 
surveyed  site  personnel  and  found  48  out  of  about  200  eligible  person- 
nel interested  in  taking  the  course. 

From  the  OOMISAT  point  of  view  it  was  questionable  that  a 
comparable  control  group  could  be  established  if  the  BE/E  course  was 
used  for  the  demonstration  since  personnel  taking  BE/E  training  are 
usually  personnel  just  entering  the  service.  However,  it  was  found 
that  thousands  of  individuals  have  taken  the  course,  and  that  occasion- 
ally Navy  personnel  are  sent  from  their  duty  station  to  take  the 
course.  Consequently,  it  was  assumed  that  the  historical  files  would 
contain  a sufficient  number  of  individuals  similar  to  the  Stockton 
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personnel  to  establish  a control  group.  It  should  therefore  be  pos- 
sible to  make  a comparison  between  the  academic  achievement  of  person- 
nel returning  to  a training  center  for  training,  and  those  receiving 
the  same  training  at  their  operational  site.  Use  of  the  BE/E  course 
would  also  permit  the  analysis  of  the  cost  differences  in  providing  the 
training  at  a training  center  and  at  the  remote  site.  With  these  ad- 
vantages, plus  the  need  for  BE/E  training,  the  CMI  supported  BE/E 
course  was  chosen  for  the  demonstration. 

The  demonstration,  in  addition  to  testing  the  delivery  of  CMI 
to  an  operational  site,  offers  the  opportunity  to  compare  the  effec- 
tiveness of  CMI  and  correspondence  courses  in  the  same  setting.  This 
will  be  done  by  having  volunteers  at  Stockton  outlying  sites  take  the 
traditional  correspondence  course  in  electricity  and  electronics.  At 
the  end  of  the  demonstration  a specially  prepared  exam  will  be  given  to 
both  CMI  and  correspondence  course  graduates  to  compare  what  they  have 
learned.  In  order  to  validate  the  exam,  it  will  also  be  given  to  a 
control  group  with  no  formal  training  in  electricity  and  electronics. 

4.  Attitude  Survey 

A basic  problem  associated  with  using  "historical  record  per- 
sonnel" as  a control  group  is  that  attitude  surveys  cannot  be  adminis- 
tered. Consequently,  it  is  not  possible  to  compare  the  control  and 
experimental  groups  relative  to  student  attitude  and  resultant  academic 
achievement.  However,  it  is  possible  to  compare  the  attitudes  of  stu- 
dents toward  CMI  per  se  and  toward  receiving  such  training  at  the  job 
site  or  at  a training  center  regardless  of  the  content.  From  a control 
group  point  of  view,  this  can  be  accomplished  by  administering  the  at- 
titude survey  to  any  student  returning  from  a job  site  to  a training 
center  to  take  any  CMI  supported  training  course.  This  approach  was 
deemed  acceptable. 
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5.  Demonstration  Schedule 

The  decision  to  use  the  Stockton  site  for  the  demonstration 
significantly  impacts  the  schedule  of  events  by  reducing  the  time 
needed  to  execute  the  various  phases. 

According  to  the  schedule  of  events  developed  in  the  feasi- 
bility study,  the  preparation  for  the  demonstration  was  to  take  place 
from  July  through  December  1977,  the  demonstration  was  scheduled  for 
January  through  June  1978,  and  the  evaluation  was  scheduled  for  August 
through  November.  However,  due  to  operational  constraints  at  Stockton, 
including  a requirement  for  a major  training  program  (COMPARS) , the 
Stockton  commander  requested  that  the  demonstration  be  initiated  in 
September  1977  and  run  through  February  1978.  This  leaves  two  months 
for  the  demonstration  preparation,  July  and  August  1977.  The  evalu- 
ation phase  would  run  from  March  through  June  1978. 

The  design  work  in  this  document  reflects  the  change  in 
schedule.  Nevertheless,  it  is  recognized  that  these  changes  may  create 
a problem  for  agencies  and  organizations  being  tasked  for  resources; 
therefore,  it  may  not  be  possible  to  keep  the  schedule  of  events  envis- 
aged in  this  report.  Every  effort  will  be  made  to  minimize  the  slip 
page. 


C.  Demonstration  Overview 

A representative  sample  of  U.S.  Navy  RM  personnel  assigned  to 
NAVCOMMSTA  Stockton  will  be  chosen  from  a group  of  volunteers.  Begin- 
ning in  September  they  will  enroll  in  the  common  core  of  the  BE/E 
Course  File  69.  They  will  spend  an  average  of  10  hours  of  watch  time 
per  watch  string  on  the  course  until  the  common  core  is  completed. 

Since  the  course  is  self-paced,  the  time-to-complete  will  vary  for  each 
student,  however,  the  maximum  time  required  to  complete  the  course 
should  be  no  more  than  20  weeks. 
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The  students  will  do  their  course  work  in  an  area  set  aside  in  the 
Fleet  Conmunications  Center,  where  learning  materials,  equipment,  and 
supervisory  personnel  will  be  located.  Periodically  they  will  interact 
with  the  Millington,  Tennessee  computer  center  through  the  use  of  an 
Opscan  12/17  optical  reader  and  a Terminet  teletype.  The  Opscan  will 
be  used  to  feed  test  answer  sheets  into  the  computer  for  grading,  and 
the  Terminet  will  be  used  for  student  feedback  providing  test  results 
and  prescriptive  information. 

During  portions  of  the  day  and  eve  watches  a Learning  Supervisor 
(LS)  will  be  available  to  address  course  and  equipment  related  ques- 
tions. When  the  LS  is  not  available,  the  senior  maintenance  elec- 
tronics technician  (ET)  will  take  over  these  responsibilities. 

Communication  between  the  student  and  the  computer  center  will 
take  place  using  a commercial  dedicated  phone  line  between  the  Fleet 
Communications  Center  at  Stockton  and  the  Management  Instructional 
Information  Systems  Activity  (MIISA)  CMI  concentrator  at  the  training 
center  at  San  Diego.  MIISA  lines  will  connect  the  training  center  at 
San  Diego  with  the  training  center  at  Millington.  Stockton  will  be 
on-line  with  the  CMI  computet  center  from  0600  to  1830  Pacific  Coast 
time,  five  days  a week.  Materials  developed  at  other  times  will  be 
saved  and  submitted  at  0600  the  next  day  on  which  the  computer  is 
operating . 

The  correspondence  course  students  at  the  outlying  Stockton  sites 
will  also  be  permitted  to  study  an  average  of  10  hours  per  watch 
string.  At  that  rate,  they  should  complete  the  course  within  16 
weeks.  After  completing  the  course,  they  and  the  CMI  course  gradu- 
ates will  take  the  special  exam  on  electricity  and  electronics. 
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D.  Organization  of  the  Report 

The  results  of  the  design  work  are  presented  in  the  following 
chapters:  Chapter  II  presents  the  economic  rationale  for  the  effort. 
Chapter  III  describes  the  research  design.  Chapter  IV  presents  the 
demonstration  design,  specifying  the  activities  required  to  conduct  the 
demonstration,  carry  out  the  research,  and  provide  support  in  terms  of 
communication  lines,  lab  equipment,  course  materials  and  personnel. 

The  formal  tasking  required  to  obtain  the  support  of  different  Navy 
organizations  is  briefly  described  in  Chapter  V.  The  master  plan  link- 
ing the  various  activities  is  given  in  Chapter  VI.  Supporting  materi- 
als are  contained  in  a series  of  appendices. 
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Chapter  II 

ECONOMIC  RATIONALE  FOR  COMISAT 


A.  Introduction 

An  economic  analysis  of  the  COMISAT  concept  was  carried  out  to 
determine  its  feasibility  as  an  operational  system.  The  investigation 
followed  the  guidelines  prescribed  by  OODI  7041.3  and  SECNAVINST 
7000. 24A  (Reference  3).  The  following  section  describes  the  key  find- 
ings of  the  analysis  and  presents  recommendations.  This  is  followed  by 
an  overview  of  the  methodology  used  in  the  analysis.  The  details  of 
the  analytical  procedures  are  presented  in  Appendices  A through  G. 

B.  Key  Findings  and  Recommendations 

1.  Reduction  in  Navy  Training  Costs 

The  analysis  shows  that  full  implementation  of  the  COMISAT 
concept  could  result  in  present  value  net  savings  of  18  to  35  million 
dollars  based  on  an  eight-year  program  and  a 10-percent  discount 
rate,^  and  depending  on  the  set  of  conditions  assumed  as  shown  in 
Figure  II-l . The  top  half  of  Table  II-l  shows  the  incremental  invest- 
ment and  savings-to- investment  (or  benefits  to  investment)  ratio  asso- 
ciated with  the  savings  for  a representative  case.  It  should  be  noted 
that  the  present  value  saving  estimates  are  probably  low  since  the 
analysis  utilized  conservative  approximations  of  the  economic  benefits 
of  COMISAT. 


Vhe  present  value  concept,  and  the  relation  of  present  value  to  annual 
costs  and  savings  are  described  in  Appendix  A. 
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Table  IT  1.  Characteristics  of  a Representative  COMISA  T 
Program 


Financial 

Present  value  net  savings  (eight 
year  program,  10%  discount) 

$29,777,000 

Annual  Savings 

$10,858,000 

Incremental  investment 

$31,582,000 

Savings-to-investment  ratio 

1.94 

Operational 

Number  of  CMI  courses 

69 

Number  of  sites 

— ships 

411 

— CONUS  land  bases 

80 

— overseas  land  bases 

47 

Number  of  students 

20,881 

Proportion  of  CMI  students 
served  at  sites 

96.7% 

These  savings  are  achieved  by  using  CMI  to  train  personnel 
who  normally  take  skill  progression  courses  at  the  training  center 
which  requires  a permanent  change  of  station  (PCS)  or  temporary  duty 
(TOY)  assignment.  The  major  source  of  COMISAT  savings  is  the  reduction 
in  travel  and  living  expenses  achieved  by  having  the  students  remain  at 
their  operational  land  site  or  ship.^  The  analysis  shows  that  on  the 


^An  explanation  of  the  calculation  and  interpretation  of  annual  savings 
can  be  found  in  Appendix  A. 
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order  of  30  percent  or  more  of  skill  progression  students  who  ordinar- 
ily travel  to  a training  center  could  be  served  by  COMISAT.^  This 
means  that  more  than  20,000  students  a year,  representing  more  than  one 
tenth  of  the  total  number  of  Navy  students  trained  per  year  would  be 
affected.  The  bottom  of  Table  II-l  lists  the  number  of  sites  and 
trainees  served  for  the  representative  case. 

The  uncertainties  which  account  for  the  variation  in  the 
above  results  have  not  been  resolved  as  discussed  in  the  next  section. 
However,  even  the  lower  values  of  savings  are  large  enough  to  warrant 
serious  consideration  by  the  Navy  of  implementing  COMISAT,  and  are  cer- 
tainly large  enough  to  justify  the  proposed  demonstration. 

These  conclusions  are  also  important  for  the  other  services. 
If  equivalent  savings  can  be  achieved  for  each  of  the  other  two  ser- 
vices, which  is  reasonable  to  expect,  then  total  Defense  Department 
savings  could  have  a present  value  of  between  54  and  105  million  dol- 
lars for  an  eight  year  program. 


2.  Uncertainties 

The  variation  in  the  savings  quoted  above  is  due  to  uncer 
tainties  in  three  areas: 

• It  is  not  known  what  percentage  of  Navy  skill  progression 
courses  are  suitable  for  conversion  to  CMI.  Values  of  60  to 
80  percent  were  examined  in  this  analysis.  A value  of  70 
percent  was  picked  as  nominal  based  on  discussions  with  Navy 
training  personnel  and  a review  of  courses.  A 10  percent  var- 
iation was  used  to  analyze  the  sensitivity  of  the  results  to 
the  nominal  assumptions. 

• The  frequency  of  terminal  breakdowns  under  operational  condi- 
tions on  land  and  at  sea  is  not  known  with  certainty.  Manu- 
facturer estimates  were  increased  by  one  third  on  land  (from 
three  to  four  per  year)  and  one  half  at  sea  (from  four  to  six 
per  year)  to  assess  the  sensitivity  of  the  results. 


^An  explanation  of  the  calculation  and  interpretation  of  annual 
savings  can  be  found  in  Appendix  A. 
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• The  amount  of  student  time  saved  by  having  CMI  instead  of 

conventional  instruction  is  estimated  between  10  and  20  per- 
cent (Reference  4).  Both  cases  were  considered  here. 

The  analysis  shows  that  the  resultant  change  in  savings  at- 
tributable to  (1)  the  change  in  annual  terminal  breakdown  rates  from 
three  to  four  on  land  and  four  to  six  on  ships,  (2)  a 10  percent  change 
in  the  proportion  of  courses  convertible  to  CMI,  and  (3)  a five  percent 
change  in  student  time  savings  with  CMI  would  each  result  in  a change 
of  about  five  million  dollars  in  present  value  savings.  That  corres- 
ponds to  an  annual  change  in  savings  of  about  one  million  dollars. 

Table  II-2  lists  the  results  for  the  two  cases  which  were 
developed  in  the  greatest  detail.  The  cases  show  the  effect  of  diffe- 
rent remote  site  terminal  breakdown  rates  on  the  savings  achieved,  the 
number  of  courses  that  can  be  delivered,  and  on  the  number  of  ships  and 
land  sites  served. 

Table  11-2.  Summary  of  Results 


Item 

Low 

Maintenance"' 

High 

Maintenance'^' 

Number  of  CMI  courses 

69 

67 

Number  of  sites 

538 

434 

— ships 

411 

328 

- CONUS 

80 

66 

— overseas 

47 

40 

Proportion  of  CMI  students  served  at  the  sites 

96.7% 

92% 

Annual  net  savings 

$10,858,000 

$10,190,000 

Present  value  of  savings  (eight  year  program) 

$29,777,000 

$26,384,000 

Incremental  Investment 

$31,582,000 

$32,096,000 

Savings-to-Investment  Ratio 

1.94 

1.82 

low  maintenance  case  assumes  that  three  breakdowns  occur  each  year  on  land  and  four  breakdowns 
occur  each  year  at  sea. 


^’the  high  maintenance  case  assumes  four  breakdowns  per  year  on  land  and  six  at  sea.  This  case  also 
assumes  that  due  to  higher  breakdown  rates  spare  terminals  will  be  required  at  all  sites  serviced  by 
Navy  personnel. 
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There  is  a decrease  of  five  million  dollars  in  present  value  savings 
for  increases  in  breakdown  rates  of  one  third  on  land  and  one  half  at 
sea.  The  greatest  impact  is  at  sea  where  83  fewer  ships  could  be 
served  with  the  higher  breakdown  rates.  The  number  of  courses  that  can 
be  delivered  decreases  from  69  to  67  because  of  the  higher  maintenance 
costs.  In  both  cases,  more  than  90  percent  of  the  students  who  nor- 
mally travel  to  a training  center  to  attend  the  feasible  CMI  courses 
would  receive  the  training  at  their  base  or  ship. 

These  two  cases  assume  that  70  percent  of  Navy  skill  progres- 
sion courses  are  operationally  suitable  for  conversion  to  CMI,  and  that 
CMI  requires  20  percent  less  time  than  conventional  instruction.  The 
effect  on  savings  of  varying  the  latter  two  assumptions  is  shown  in 
Table  II-3.  A 10  percent  change  in  the  number  of  courses  suitable  for 
CMI  results  in  a change  in  the  present  value  of  the  savings  of  abcut 
five  million  dollars. 

Table  IT3.  Sensitivity  of  Present  Value  Savings  to  Time  Savings  and  CMI 
Convertibility  Assumptions:  Low  Maintenance  Case 


Percentage  of  Skill  Progression 
Courses  Convertible  to  CMI 

Percentage  of  Student  Time  Saved 

10% 

15% 

20% 

60% 

$17,627,000 

$21,326,000 

$25,025,000' 

70% 

$21,770,000 

$26,085,000 

$30,401,000 

80% 

$25,913,000 

$30,845,000 

$35,777,000 

'The  values  contained  in  Table  11-3  are  estimates  based  on  a closed  form  expression  for  net  COMISAT  cost. 
Hence,  this  number  differs  slightly  from  the  present  value  savings  figure  in  Table  11-2. 
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The  effect  of  a change  in  student  time  savings  due  to  the  use 
of  CMI  is  greater  when  there  is  a higher  proportion  of  courses  suitable 
for  conversion.  When  60  percent  of  the  skill  progression  courses  are 
suitable  for  CMI,  a 10  percent  change  in  student  time  savings  (from  10 
to  20  percent)  changes  the  present  value  savings  by  $7,398,000  (from 
$17,627,000  to  $25,025,000).  However,  when  80  percent  of  the  courses 
are  convertible,  the  same  change  in  student  time  savings  changes  the 
present  value  of  savings  by  $9,864,000  (from  $25,913,000  to 
$35,777,000) . 

The  lowest  estimate  of  present  value  savings  is  14  million 
dollars,  assuming  high  maintenance,  60  percent  course  convertibility  to 
CMI,  and  10  percent  student  time  savings  with  CMI. 

The  savings  results  are  sensitive  to  other  assumptions,  but 
not  to  the  same  extent  as  those  just  discussed.  For  example,  the  re- 
sults in  Tables  II-l  to  II-3  assume  that  land  sites  within  30  miles  of 
a central  location  can  be  served  by  one  terminal  at  that  location. 
Courier  service  would  carry  inputs  to  the  terminal  and  return  messages 
to  the  base.  If  that  is  not  feasible,  present  value  savings  would  be 
reduced  by  one  million  dollars. 


3.  Communication  Requirements 

Another  concern  is  for  the  availability  of  communications 
lines.  As  the  number  of  CMI  students  increases,  the  capacity  required 
will  increase  the  Navy  communications  required,  particularly  from  the 
CMI  computer  to  the  remote  sites.  To  assess  this,  the  capacity  re- 
quired was  estimated^  based  on  the  characteristics  of  the  CMI  mes- 
sages. The  total  communications  load  at  the  average  site  is  estimated 
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to  be  0.28  characters  per  second,  and  at  the  CMI  computer,  150  charac- 
ters per  second.  This  would  appear  to  be  a reasonable  load  for  the 
AUTODIN  II  system. 

4.  Recommendations 

Since  the  economic  analysis  shows  large  potential  savings,  it 
is  important  to  test  a major  assumption,  i.e.,  that  the  COMISAT  concept 
is  workable  at  a remote  site.  It  is  necessary  to  show  that  supporting 
personnel  assignments  can  be  carried  out  as  assumed  for  the  analysis, 
that  students  can  take  the  time  on  site  for  training,  and  that,  under 
those  conditions,  the  training  is  effective.  The  demonstration  pro- 
posed in  this  document  would  help  confirm  that  the  concept  is  practical 
and  desirable. 


Additional  investigation  of  the  economic  benefits  of  COMISAT 
should  be  conducted: 

• The  Navy  courses  should  be  surveyed  in  detail  to  determine 
suitability  for  CMI,  enrollments,  and  lengths. 

• A better  approximation  of  equipment  breakdown  rates  should  be 
obtained.  In  the  event  that  these  rates  are  higher  than 
those  in  the  high  maintenance  case  explored  here,  the  manu- 
facturer should  be  consulted  in  an  attempt  to  improve  main- 
tainability characteristics. 

• An  investigation  should  be  made  to  determine  whether  the 
average  travel  and  living  expenses  used  in  this  analysis  ade- 
quately reflect  the  range  of  costs  and  the  usual  payment 
procedures. 

• The  availability,  cost,  and  feasibility  of  courier  service 
between  nearby  sites  should  be  investigated  to  obtain  an  im- 
proved estimate  of  the  extent  to  which  clustering  of  sites  is 
practical  and  profitable. 

• The  available  capacity  of  AUTODIN  at  each  site  should  be  as- 
sessed to  insure  that  the  additional,  demand  due  to  COMISAT 
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can  be  adequately  met.  In  particular,  the  AUTODIN  II  capa- 
city to  the  CMI  oorputer  will  have  to  be  laz^e  enou^i  to  ac- 
oonrodate  the  new  load.l 

C.  Background  and  ^^roach 

1.  Background 

There  are  Navy  courses  in  self-paced  instruction  format,  sore 
of  which  are  already  progranmed  onto  the  CMI  oonputer  at  Matf^is  and  avail- 
able for  CMI  instruction  at  locaticxis  such  as  Memphis,  Great  Lakes,  San 
Diego,  and  Orlando.  In  the  future,  CNET  intends  to  ej^jand  the  CMI 
capability  by; 

• Develc:p>ing  and  prograitming  additional  courses  into  CMI 

• Enlarging  the  CMI  conputer  to  acconnodate  the  increased 
course  and  student  loads 

It  is  possible  that  the  COMISAT  project  could  play  a significant  role  in 
these  future  plans.  The  need  for  an  additicxial  training  center  and  the 
operations  of  the  existing  training  centers  might  be  impacted. 

The  benefits  vAiich  could  be  expected  fron  an  implementation  of  OQMISAT 
include  those  attributable  to  CMI  and  those  attributable  to  remote  site 
tj  aining.  Navail  eiperience  (Reference  1)  indicates  that  utilization  of 
CMI  at  a training  center  yields: 

• A reduction  in  course  time 

• A reduction  of  instructional  and  svpport  personnel 


^The  net  cost  of  these  new  lines  was  assvmed  to  be  the  same  as  the 
additional  oortmunications  cost  for  the  CMI  system  enlarged  to  include 
these  skill  progression  courses. 
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• An  increase  in  student  end-of-course  achievement  levels 

• A reduction  in  student  attrition. 

These  benefits  could  also  be  expected  from  CMI  at  operational  Navy 
sites.  In  addition,  COMISAT  would  allow  the  student  to  remain  on  the 
job  while  training,  thus  resulting  in: 

• A saving  in  travel  and  living  cost 

• Training  for  personnel  unable  to  leave  the  operational  site  • 

• An  increase  in  the  availability  of  key  personnel  to  handle 
contingencies  at  the  site 

The  elements  of  cost  and  saving  involved  in  achieving  the 
above  benefits  have  never  been  completely  defined.  Determination  of 
important  cost  elements  was  a necessary  step  in  the  analysis  procedure. 

Table  II-4  contains  an  overview  of  the  elements  of  cost  and 
saving  used  in  this  analysis  and  attributable  to  the  COMISAT  system  as 
compared  with  the  convent ional/CMI  system.  Certain  courses  can  be  eco- 
nomically justified  for  CMI  development  by  savings  at  training  centers. 
The  only  cost  to  COMISAT  of  utilizing  these  courses  at  remote  sites  is 
the  cost  of  the  site  terminals  required.  Savings  achieved  by  offering 
these  courses  through  COMISAT  include  student  travel  and  living  costs 
and  the  reduction  in  terminal  cost  at  training  centers.  Other  courses 
could  not  be  economically  justified  for  use  only  at  training  centers, 
but  could  be  justified  by  savings  achieved  through  COMISAT.  For  such 
courses  the  costs  to  COMISAT  include  not  only  the  site  terminal  cost 
but  also  the  CMI  course  developMnent,  coding  and  maintenance  costs,  and 
computer  leasing  costs.  Also,  additional  terminals  required  to  teach 
these  courses  at  the  training  center  to  students  who  cannot  be  reached 
through  COMISAT  must  be  considered  under  COMISAT  costs.  The  savings 
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Tad/e  11-4.  Summary  of  COMISA  T Costs  and  Savings  Elements 

1 

h- 

C*" 

Courses  Justified  For 
Training  Center  Use 

Additional  Courses  Justified 
Only  By  COMISAT 

Costs  for 
COMISAT 

Terminal  cost  at  sites 

Terminal  cost  at  addi- 
tional sites 

Course  development,  coding 
and  maintenance 

Computer  leasing 

Training  center  terminals 

Savings  due  to 
COMISAT 

Travel  and  living  costs  saved 
by  receiving  training  at  the 
operational  land  base  or  ship 

Time  and  living  costs  sav- 
ed by  CMI  over  conven- 
tional instruction 

Reduced  number  of  train- 
ing center  terminals 

Travel  and  living  costs  saved 
by  receiving  training  at  the 
operational  land  base  or  ship 

credited  to  COMISAT  for  these  courses  consist  of  the  student  travel  and 
living  expenses  eliminated  by  training  at  the  site,  and  time  and  living 
savings  gained  through  decreased  student  training  time. 

The  analysis  of  these  cost  elements  consisted  of  four  steps: 

• Determine  courses  economically  feasible  for  CMI  at  training 
centers 

• Determine  economically  feasible  CXJMISAT  sites  for  delivery  of 
the  CMI  courses  justified  for  use  at  training  centers 

• Using  the  sites  identified  in  the  second  step,  determine  ad- 
ditional CMI  courses  justified  by  COMISAT  savings 

• For  the  complete  set  of  training  center-justif ied  and 
COMISAT-justified  CMI  courses,  determine  the  number  of 
COMISAT  sites  for  which  savings  are  maximized. 

Each  of  these  steps  are  discussed  in  detail  in  the  next  section. 
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2.  The  Approach 

a.  Determination  of  Courses  Economically  Feasible  for  CMI 
at  the  Training  Centers 

(1)  The  Courses 

The  first  step  taken  in  analyzing  courses  for  CMI 
development  was  to  obtain  data  on  overall  training  enrollments  and  in- 
dividual course  enrollments.  The  necessary  data  on  annual  overall 
training  loads  is  summarized  in  Table  II-5  (Reference  S)^. 

Six  distinct  training  segments  are  identified  by 
this  data.  There  are  three  categories  of  courses  offered:  initial 
skill  training,  skill  progression  training,  and  functional  training. 
Initial  skill  training  consists  of  Class  A courses  offered  immediately 
after  recruit  training.  Skill  progression  training  is  made  up  of  more 
advanced  Class  C type  courses,  offered  to  sailors  who  have  already  had 
some  work  experience  and  are  preparing  themselves  for  an  advanced  rat- 
ing. Finally,  functional  training  consists  of  short  duration  training 
of  shipboard  duties  given  at  the  ports  of  fleet  concentration  such  as 
San  Diego. 


There  are  also  two  types  of  students:  those  on 
permanent  change  of  station  (PCS)  and  those  on  temporary  duty  (TDY) . 
Those  on  permanent  change  of  station  include  recruits  going  from  re- 
cruit training  to  a training  center  and  then  to  fleet  or  shore  sta- 
tions, and  sailors  going  from  fleet  or  shore  stations  to  a training 
center  and  then  to  new  duty  stations.  Temporary  duty  occurs  when  the 
sailor  leaves  his  station  for  a short  period  of  time,  such  as  to  go  to 
a training  center,  and  then  returns  to  his  permanent  station. 

The  most  likely  candidates  for  COMISAT  instruction 
are  those  taking  skill  progression  training.  The  students  in  these 

^See  Appendix  C for  original  data. 
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Table  ITS.  Navy  FY  78  Training  Loads 


Category 

PCS 

TDY 

ToUl 

Initial  Skill  Training 

1 Entrants 

157,833 

6,269 

164,102 

2.  Graduates 

156,073 

6,214 

162,287 

3.  Average  Course  Length  (days) 

42 

42 

42 

4.  Loads  Average  on  Board  (AOB) 

19,152 

570 

19,722 

5.  Percentage  of  Loads’  ’ ’ 

57% 

2% 

59% 

Skill  Progression  Training 

1 Entrants 

30,042 

32,867 

62,909 

2.  Graduates 

29,336 

32,330 

61 ,666 

1 3.  Average  Course  Length  (days) 

51 

51 

51 

^ 4 Loads  Average  on  Board  (AOB) 

7,274 

2,526 

9,800 

5.  Percentage  of  Loads”  * 

22% 

8% 

30% 

1 Functional  T raining 

1 Entrants 

22,226 

329,340 

351,566 

2.  Graduates 

21,611 

320,619 

342,230 

3 Average  Course  Length  (days) 

4 

4 

4 

4 Loads  Average  on  Board  (AOB) 

1,224 

2,863 

4,097 

5.  Percentage  of  Loads’” 

4% 

9% 

13% 

Grand  Total 

1.  Entrants 

2.  Graduates 

3.  Average  Course  Length  (days) 

4.  Loads  Average  on  Board  (AOB) 

5.  Percentage  of  Loads’” 

578,577 

566,183 

33,619 

100’2' 

(1)  Percentage  of  loads  * percentage  of  total  CNET  population  enrolled 

(2)  A 2%  error  due  to  roundoff  is  found  if  the  individual  numbers  are  added 


groups  are  productively  employed  before  their  training  and  could  con- 
tlnue  to  carry  out  their  duties,  although  at  a reduced  level,  if  they 
were  to  receive  training  on  board  ship  or  at  a land  base.  The  group 
taking  initial  skill  training  on  TDY  was  not  included  in  the  analysis 
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even  though  it  might  actually  benefit  from  COMISAT,  because  it  is  rela- 
tively small  and  data  on  initial  skill  courses  are  not  readily  avail- 
able for  use  in  this  analysis.  Initial  skill  students  on  PCS  are  not 
likely  candidates  for  CMI  because  their  lack  of  skills  makes  them  of 
little  use  while  training  at  the  site.  In  this  regard,  operational 
commanders  have  been  found  to  oppose  the  idea  of  placing  untrained  men 
at  their  sites  or  on  board  their  ships.  Finally,  functional  training 
segments  are  not  considered  in  the  study  since  this  instruction  is  pri- 
marily port-side  team  training  and  is  essentially  delivered  to  the 
remote  site  already.  Hence,  in  this  analysis,  the  only  students  con- 
sidered as  candidates  for  COMISAT  are  those  in  the  skill  progression 
groups. 


Having  identified  the  candidate  training  segment, 
the  next  step  is  to  obtain  data  on  course  enrollments.  The  only  ob- 
tainable data  on  individual  enrollments  for  skill  progression  courses 
are  the  title,  number  of  graduates,  and  course  length  for  each  of  the 
ten  highest  volume  skill  progression  courses  (Reference  S) . Average 
attendance  and  student  man-days  have  to  be  estimated.^  The  derived 
enrollment  information  for  these  10  courses  is  given  in  Table  II-6. 

Enrollment  data  for  additional  courses  are  gener- 
ated by  fitting  a curve  through  the  known  populations  for  the  10  high 
est  volume  courses  and  extrapolating.^ 


With  all  data  on  enrollment  and  course  length  com- 
plete, the  courses  eligible  for  conversion  to  Oil  are  identified  by 
calculating  and  comparing  costs  and  savings  to  determine  those  which 

^See  Appendix  D. 
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Table  ITS.  Ten  Specialized  Skill  Courses  Producing  Most  Graduates 


Title 

Number 
of  Grads 

— 

Average 

Attendance 

Length 

(Calendar 

Days) 

Student 

Man-Days 

1.  Instructor  Basic 

3,110 

3,141 

24 

75,384 

2.  Career  Information  Counselor 

1,323 

1,336 

26 

34,736 

3.  "Nuclear  Propulsion  Plant  Operator  Mechanical 

1,174 

182 

4.  Sonar  Electronics  Intermediate 

960 

970 

117 

113,490 

5.  Air  Conditioning  and  Refrigeration 

611 

617 

53 

32,701 

6.  "Nuclear  Propulsion  Plant  Operator  Electrical 

608 

182 

7 International  Morse  Code 

586 

592 

82 

48,544 

8.  "Nuclear  Propulsion  Plant  Operator  Reactor  Control 

512 

182 

9.  Marine  Gas  Turbine  Basic 

401 

405 

40 

16,200 

10.  Surface  Explosive  Ordnance  Disposal  Refresher 

400 

404 

28 

11,312 

Total  ( 10  Courses) 

9,685 

Total  (7  Courses) 

7,391 

332,367 

*Not  suitable  for  CMI  format. 


have  a net  savings.^  The  following  two  sections  describe  the  proce- 
dures and  assumptions. 

(2)  CMI  Costs  at  the  Training  Centers 

The  costs  associated  with  the  transfer  of  tradi- 
tional courses  to  CMI  for  training  center  use  include  development  and 
coding,  course  maintenance,  computer  leasing,  and  terminal  purchase  and 
maintenance.  Before  a course  can  be  made  available  on  CMI,  it  must  be 
developed  into  the  proper  CMI  format  and  coded  for  the  CMI  computer. 
Assuming  the  materials  are  less  than  10  percent  audio  visual,  the  de- 
velopment cost  is  estimated  to  be  $2,930  per  hour  of  instruction 
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See  Appendix  E. 
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(Reference  6 ; • The  cost  of  coding  is  between  $200  and  $300  per  in- 
struction hour,  and  a mean  coding  cost  of  $250  per  hour  is  used  in  the 
calculations.  The  estimated  total  development  and  coding  cost  per  hour 
is  $3,180.  However,  since  CNET  is  planning  to  redevelop  courses  for 
some  70  to  80  ratings  in  the  next  4 to  7 years,  those  courses  would 
cost  $1,130  per  hour  even  if  they  are  not  developed  for  CMI . This  re- 
sults in  an  incremental  cost  per  hour  of  training  for  developing  a CMI 
course  of: 

$3,180  - $1,130  = $2,050. 

It  was  assumed  that  by  expending  five  percent  of  the  incremental  de- 
velopment and  coding  costs  ($2,050  per  hour  of  training)  each  year,  the 
CMI  program  can  be  adequately  maintained  for  12  years  (Reference  6) . 

For  an  average  course  of  255  hours,  annual  course  maintenance  is 

(255)  ($2,050)  (.05)  = $26,137. 

The  computer  leasing  costs  have  two  aspects 
(Reference?);  expansion  of  the  computer  main  frame,  and  expansion  of 
the  number  of  peripherals,  particularly  the  disk  packs.  The  current 
main  frame  accommodates  6,000  average  on  board  (AOB) . The  planned  ex- 
pansion to  a Model  60  computer  will  have  a maximum  capacity  of  16,000 
AOB  students.  It  is  assumed  that  this  expansion  would  accommodate  the 
additional  skill  progression  courses.  If  the  assumption  is  not  true, 
the  computer  could  be  expanded  to  a Model  80,  at  additional  cost.  With 
the  current  mix  of  courses  on  the  computer,  one  disk  pack  would  be  re- 
quired for  each  additional  1,000  AOB  students,  assuming  an  average  of 
one  student  response  per  hour.  This  would  cost  $800  per  month.  How- 
ever, if  the  volume  of  students  per  course  decreased,  as  it  would  if 
many  additional  courses  were  added,  the  requirement  might  be  more  than 
one  disk  pack  per  1,000  students.  For  purposes  of  this  analysis,  it  is 
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assumed  that  an  additional  1.5  disk  packs  would  be  required  for  each 
1,000  students.  Note  that  the  final  results  have  low  sensitivity  to 
the  required  number  of  disk  packs  so  that  the  relatively  arbitrary  na- 
ture of  the  figure  used  is  of  little  consequence.  Thus,  the  annual 
cost  of  disk  packs  for  each  1,000  AOB  students  is  approximately 

($800)  (12)  (1.5)  = $14,400. 

Since  the  computer  mainframe  will  be  expanded  whether  or  not  it  is  used 
for  CMI,  no  computer  expansion  costs  are  incurred  by  development  of 
courses  into  CMI.  Therefore,  the  only  computer  cost  involved  is  the 
leasing  cost  of  the  disk  packs. 

The  purchase  of  terminals  is  cheaper  than  leasing, 
based  on  the  data  in  Table  11-1.  The  purchase  price  for  the  terminal 
cluster  is  $14,250  and  the  annual  maintenance  is  $1,764.  The  present 
value  of  the  maintenance  over  eight  years  is  $9,873. 

One  cluster  is  usually  required  for  each  60  stu- 
dents, and  it  is  assumed  that  clusters  could  be  shared.  Therefore  a 
proportional  cluster  cost  is  assigned  to  each  course  on  the  basis  of 
the  AOB  student  load.  A course  with  30  students  would  be  charged  half 
the  cost  of  the  cluster  under  those  circumstances. 

(3)  CMI  Savings  at  the  Training  Center 

The  savings  achievable  by  implementing  CMI  courses 
at  a training  center  accrue  for  the  most  part  from  reduced  student  pay 
and  living  costs  due  to  decreased  training  time.  There  are  other 
sources  of  savings,  such  as  reduced  instructor  time,  but  those  are  not 
considered  significant  enough  to  justify  the  additional  effort  required 
to  include  them  in  the  analysis.  Hence  the  savings  identified  here  are 
conservative  estimates. 
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Table  IT  7,  Naval  Training  Center  CM!  Cluster  Equipment  and  Associated  Costs 


VI 


Totals 


$14,250  -t  S147/mo. 


Item 

Purchase  Option 

Lease  Option 
(per  month) 

Investment 

Maintenance 

1 OPSCAN  17  (Basic  Terminal) 

$ 8,998 

$261 

Maintenance 

$ 69/mo. 

69 

1 - OPSCAN  Automatic  Feed 

652 

27 

Maintenance 

5/mo. 

5 

1 - Terminet  1200 

4,200 

132 

Maintenance 

68^mo. 

68 

1 - GDC202-9D  Modem 

400 

13 

Maintenance 

5/mo. 

5 

$580 /mo. 


Presearch  Corporation's  analysis  of  eight  courses 
that  underwent  Instructional  System  Development  (ISD)  type  development 
shows  an  average  saving  of  20  percent  of  student  man-hours  of  atten- 
dance time,  as  compared  with  the  time  required  using  traditional  in- 
struction (Reference  4) . This  planning  factor  is  used  throughout  the 
analysis,  and  the  sensitivity  of  the  results  to  the  parameter  is  tested 
as  described  in  Section  B.  Using  the  20  percent  student  time  saving 
factor,  the  man-hour  savings  for  each  course  are  converted  into  equiva- 
lent student  salary  savings.^ 

While  attending  a training  center  course  a student 
is  provided  food  at  the  same  cost  to  the  Navy  as  at  his  operational 
site.  However,  he  or  she  is  also  given  $2.50  per  day  in  cash  to  cover 
miscellaneous,  out-of-pocket  living  expenses.  Due  to  the  reduced 


^See  Appendix  E. 


Coum 

Nyniter 

If 

III 

Gauraa 

tima 

(bom) 

Avorapa 
•tudanti 
on  board 
(AOB) 

Inaaatmawt  1 
coat  (eouraa 
daaaiopinant  | 
and  coding) 

Invaatmant 

Coat  (training 
oantar  tanninaia) 

158 

^3 

Total 

annual 

coat 

^awM 

valiiadf 

anfwal 

1 

3,110 

3,141 

120 

206.5 

$ 246,000 

$49,044 

$8.07 1 

$123,000 

$2,974 

$21 ,345 

$118,481 

7 

1,323 

1,336 

130 

95.18 

226.600 

22,606 

2,798 

13,325 

1.371 

17,494 

97311 

4 

960 

970 

585 

310.8 

1,199,250 

73315 

9,138 

59,962 

4,476 

73,576 

411M 

5 

611 

617 

266 

89.60 

543,250 

16,530 

2J70 

27,162 

1390 

30,823 

172311 

7 

586 

592 

410 

138.0 

840,500 

32,776 

4367 

42,025 

1,915 

47,997 

268;Ba 

9 

401 

405 

200 

44.38 

410.000 

10,540 

1306 

20,500 

639 

72.444 

125311 

10 

400 

404 

140 

30.99 

287,000 

7360 

911 

14,350 

446 

15.707 

87 

11 

447 

415 

255 

63.08 

522,760 

14361 

1356 

26,137 

908 

28,901 

161,73 

12 

427 

431 

255 

60.26 

522,760 

14,312 

1,772 

26,137 

868 

28.777 

161 

13 

409 

413 

256 

57.71 

522.750 

13,706 

1397 

26,137 

831 

28,666 

16034 

14 

393 

397 

256 

55.46 

522,750 

13,172 

1331 

26,137 

799 

28,667 

160J8 

IS 

379 

383 

258 

53.49 

622,750 

12,704 

1373 

26.137 

770 

28,481 

159,40 

16 

366 

370 

256 

51.65 

522,750 

12,267 

1318 

26,137 

744 

28,399 

15838 

17 

355 

369 

266 

65.1 

522,760 

11396 

1.473 

26,137 

721 

28,331 

158.8* 

18 

343 

346 

256 

48.4 

522,750 

11,495 

1,423 

26,137 

697 

28,257 

158,18 

Total  of  top 

7 coucaaa 

7,391 

910.45 

Total  naxt  7 
eouraa 

2,778 

391.75 

Naxt  7 eouraa 

70%  of  total 

1.943 

274.22 

Grand  total  (top 
7 plut  70%  of 
naxt  7) 


9,334 


1.1S4,67 
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Tabie  U-8.  Analysis  of  High  Volume  Skill  Progression  Courses 


8,180.044 

5,564.773 

3.896.341 
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mluaof 
total  coats 

Value  of 
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mund/vf- 

Total 

Kvini  coat 
saaintii/ 

yaof 

111 

_ J 

$123,000 

$21,345 

$119,468 

$ 414,512 

$416,491 

$37,694 

$464,185 

13.325 

ijri 

17.494 

97316 

387,021 

191.041 

17,370 

309311 

58,962 

4,476 

73.576 

411306 

1,684,870 

626,745 

56,721 

683366 

27,162 

1360 

30,823 

172,516 

732,296 

180397 

10,362 

187349 

42,025 

1,915 

47,997 

268339 

1,141.914 

268,130 

24,266 

92396 

20,500 

639 

22,444 

125.619 

546,159 

89303 

8,099 

07303 

14,350 

446 

15,707 

87312 

382,272 

62,496 

5,666 

68,162 

26.137 

908 

28,901 

161,739 

699.490 

127307 

11,512 

108,719 

26.137 

868 

28,777 

161,065 

698.127 

121316 

10,997 

132313 

26,137 

831 

28,666 

160,444 

696,900 

116393 

10,632 

126326 

26.137 

799 

28,667 

159,881 

696,81 1 

111340 

10,121 

121361 

26,137 

770 

28,481 

159,408 

694,862 

107366 

9.762 

117318 

26,137 

744 

28,3M 

158.349 

693,966 

104,166 

9,426 

1,136382 

26,137 

721 

28,331 

168,667 

693,218 

101326 

9,143 

110,169 

26,137 

807 

28,257 

168.164 

692.399 

97311 

8333 

106,441 

$2,542,073 

1,171,514 

3325,369 

1,102383 

1336.530 


11,206.066 

4321,742 

3,376,219 


2,166314 

825334 


2326372 

465362 
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training  time,  20  percent  of  these  living  expenses  are  saved  for  each 
course . 


The  total  costs  and  savings  computed  in  this  manner 
are  discounted  over  eight  years.  Since  the  development  and  coding  in- 
vestment produces  a course  with  a lifetime  of  twelve  years,  the  dis- 
counted residual  value  of  development  and  coding  is  treated  as  a credit 
to  COMISAT.  Therefore  the  resulting  present  values  consistently  repre- 
sent an  eight  year  program.^  As  Table  II-8  indicates,  this  technique 
identifies  the  top  17  courses  as  feasible  for  CMI  development.  How- 
ever, three  of  the  first  ten  courses  are  nuclear  training,  and  it  is 
assumed  that  these  would  not  be  operationally  practical  for  CMI.  This 
same  planning  factor  is  applied  to  courses  11  through  17;  so,  on  the 
average,  only  70  percent  of  these  courses  are  considered  to  have  poten- 
tial for  CMI  development  and  only  70  percent  of  these  net  savings  are 
credited  to  CMI.  The  analysis  of  the  sensitivity  of  the  results  to  the 
value  of  this  parameter  is  discussed  in  Section  B. 

b.  Determination  of  Feasible  COMISAT  Sites 

Implementation  of  CMI  at  the  training  centers  results  in 
net  savings.  COMISAT  provides  an  opportunity  for  additional  savings  by 
delivering  the  courses  to  operational  larid  bases  and  ships,  and  elimi- 
nating travel  and  living  expenses  of  the  students.  Certain  costs  and 
savings  occur  to  the  same  degree  through  both  COMISAT  and  CMI  at  the 
training  center.  Such  elements  do  not  aid  in  differentiating  between 
the  two  alternatives  and  were  not  explicitly  calculated  in  this  analy- 
sis. Hence,  the  cost  elements  discussed  below  consist  entirely  of  dif- 
ferentiating costs  and  savings. 


^See  Appendix  A for  a description  of  residual  value  and  the  method  of 
computing  its  present  value. 


33 


TAEG  Report  No.  49 


(1)  Costs  to  COMISAT 

The  costs  to  COHISAT  consist  of  the  purchase  or 
leasing  price  of  the  terminals,  the  cost  of  annual  maintenance,  and  the 
cost  of  software  changes  to  the  CMI  computer,  required  to  interface  with 
AUTODIN.  The  required  equipment  and  its  costs  can  be  found  in  Table 
II-9  (Reference  8).  Each  terminal  was  assumed  to  have  a lifetime  of 
appproximately  eight  years.  Since  leasing  over  this  period  is  much 
more  expensive  than  purchase,  only  the  purchase  option  is  considered  in 
this  analysis.  Quantity  discounts  are  available  as  found  in  Table 
II-IO  (Reference  8).  The  maintenance  costs  in  Table  II-7  are  for  sites 
in  the  continental  United  States  (CONUS)  50  miles  or  less  from  an 
Opscan  service  location.  Other  CONUS  sites  may  be  maintained  either  by 
Opscan  or  by  specially  trained  Naval  electronic  technicians  (ET's). 
Overseas  and  ship  sites  must  be  serviced  by  ET's. 


Table  11-9.  COMISA  T Terminal  Equipment  and  Cos! 


Purchase  Option 

Lease  Option 

Item 

Investment  Maintenance 

(per  month) 

1 - OPSCAN  17  (Basic  Terminal) 

$ 8,998 

$369 

Maintenance  of  one 

$69 

69 

1 - OPSCAN  Automatic  Feed 

652 

27 

Maintenance 

5 

5 

1 - Device  273  ' 

1,620 

74 

1 - LRC  Character 

180 

Maintenance  of  total  equipment 

6 

6 

Total 

$11,450  + $80/mo. 

$550 'mo. 

Spares' 

$ 6,529 

"'Required  for  compatibility  with  AUTODIN 

'^'Sased  upon  list  of  spares  recommended  by  OPSCAN  These  spares  are  required  for  all  ship  sites  and  land  sites 
yyhere  a terminal  is  maintained  by  Navy  personnel 
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Table  IT  10.  OPSCAN  Discounts  from  GSA 
Schedule 


The  cost  of  service  by  Opscan  personnel  for  sites 
greater  than  50  miles  from  the  service  center  (Reference  7*  consists  of; 

• Travel  costs  of  18C  per  mile 

• Labor  cost  of  $30  per  hour,  including  travel  time 

• An  approximate  annual  parts  cost  of  $600  obtained  from 
Opscan 's  price  for  a service  warranty 

Labor  cost  was  calculated  assuming  a mean  time  to 
repair  of  2.5  hours  per  breakdown. 

Service  by  an  ET  involves  the  following  costs; 

• An  approximate  labor  cost  of  $75  per  breakdown,  assuming  the 
same  total  (Jpscan  labor  cost  per  breakdown  as  for  Opscan 
maintenance  personnel 

• An  approximate  annual  parts  costs  of  $400  per  year  based  on 
the  price  of  a service  warranty  with  a 30  percent  return  on 
investment  removed 

• A spare  parts  inventory  which  can  be  purchased  at  an 
undiscounted  base  price  of  $6,529 


35 


V' 


I 


TAEG  Report  No.  49 

The  Opscan  maintenance  cost  on  a time  and  materials 
basis  is  less  than  the  cost  of  the  Navy  maintaining  the  equipment  it- 
self for  CONUS  sites  less  than  90  miles  from  an  Opscan  service 
center. i Five  categories  of  sites,  each  with  different  maintenance 
costs,  are  derived  in  the  analysis.  These  categories  and  their  eight 
year  discounted  maintenance  costs,  assuming  three  breakdowns  per  year, 
can  be  found  in  Table  II-ll. 


Table  II-  7 7 Site  Categories  and  Present  Value  Maintenance  Costs  for  the  Low 
Maintenance  Case 


(2)  CtXISAT  Savings 

The  COMISAT  savings  which  were  considered  in  this 
analysis  consist  of  student  travel  costs  to  and  from  the  training  cen- 
ters, student  living  costs  while  training  at  the  training  centers,  and 
training  center  terminals  which  would  no  longer  be  required.  Travel 
cost  is  assessed  differently  for  PCS  students  then  for  TOY  students. 
The  travel  cost  for  a student  attending  training  on  TOY  includes  round 


See  Appendix  F. 
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trip  travel  from  site  to  training  center  and  meals  required  during  tra- 
vel time.  One-way  travel  expenses  from  site  to  training  center,  plus 
meals  during  travel  time,  are  required  for  a student  attending  training 
on  PCS.  Also,  when  a student  on  PCS  travels  to  attend  a course  of  un- 
der 140  days  duration,  he  is  permitted  transfer  of  household  effects  at 
a cost  of  $291.  This  amount  increases  if  the  time  is  greater  than  140 
davs . 

Practically  all  travel  cost  is  a function  of  the 
distance  between  the  original  station  and  the  training  center  loca- 
tion. Since  the  exact  mileage  distribution  is  difficult  to  obtain, 
SUPERS  uses  an  average  distance  of  1,500  miles  for  planning  purposes. 
This  factor  is  used  here.  The  mileage  data  is  combined  with  the  cur- 
rent schedule  of  airline  tariffs,  obtained  from  the  Civil  Aeronautics 
Board  (CAB).^  Thus  the  one-way  cost  of  a 1,500  mile  trip,  including 
tax,  is  $137.  The  total  travel  expense  which  could  be  saved  if  the  en- 
tire student  population  could  be  trained  at  remote  sites  is  $3,374,054 
per  year. 

If  OOMISAT  is  employed,  there  is  also  a saving  of 
$2.50  per  calendar  day  of  training  for  incremental  living  expenses 
which  would  not  have  to  be  spent  at  a training  center.  Thus  the  incre- 
mental living  cost  saved  for  the  entire  student  population  is  $862,920 
per  year. 

The  total  gross  travel  and  living  savings  per  year 
(TGS)  for  training  at  remote  sites,  as  compared  with  CMI  training  at 
the  training  center,  are 

TGS  - $4,236,974 
^See  Appendix  G. 
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or  a present  value  over  eight  years  of 
TGSp^  = $23,714,343. 

The  number  of  training  center  terminals  needed 
would  be  reduced  if  some  students  were  trained  at  their  work  site.  One 
training  center  terminal  is  required  for  every  60  AOB  students.  Hence, 
there  is  a saving  of  one  training  center  terminal  for  each  60  AOB  stu- 
dents reached  through  COMISAT.  A purchase  price  of  $14,250  and  a main- 
tenance cost  of  $147  per  month  would  be  saved  by  eliminating  a training 
center  terminal,  as  found  in  Table  II-7.  As  indicated  earlier,  the 
purchase  option  is  the  least  expensive  and  is  used  to  estimate  the 
savings , 


The  CX)MISAT  costs  and  savings  for  each  category  are 
utilized  to  determine  a break-even  site  population.^  The  distribu- 
tion of  stuudents  over  Navy  sites  cannot  be  obtained;  hence,  it  is 
assumed  to  be  identical  to  the  enlisted  man  distribution  (Reference 
7).  A comparison  of  the  break-even  population  for  a category  and  each 
site  population  in  that  category  determine  which  sites  could  be  cost 
effectively  developed  for  COMISAT.  Sites  with  populations  greater  than 
the  break-even  value  qualify. 

c.  Determination  of  Courses  Justified  by  COMISAT  Savings 

The  next  step  is  to  derive  an  estimate  of  the  number  of 
feasible  courses,  assuming  the  number  of  sites  remains  fixed  at  the 
level  determined  previously.  To  this  end,  the  costs  and  savings  due  to 
COMISAT  are  reexamined. 


1 
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Since  only  the  feasible  courses  from  the  first  17 
courses  can  be  economically  developed  for  CMI  independently  of  COMISAT, 
the  development  and  coding,  course  maintenance,  computer  leasing  costs 
and  the  student  time  and  living  savings  for  these  courses  are  not  at- 
tributed to  COMISAT.  However,  the  development  of  any  courses  beyond 
the  17th  would  not  be  feasible  without  COMISAT,  for  these  courses  are 
justified  only  by  savings  achieved  at  remote  sites.  For  this  reason, 
the  development,  coding,  course  maintenance,  computer  leasing  costs, 
and  student  time  and  living  savings  due  to  the  use  of  CMI  for  the  18th 
and  higher  courses  must  be  attributed  to  COMISAT. 

One  analytical  concern  should  be  noted  for  considering 
costs  and  savings  associated  with  the  18th  - Nth  courses.  As  before, 
all  cost  elements  are  considered  over  an  eight  year  period.  It  is  also 
important  to  note  that  the  annualized  values  not  only  represent  costs 
and  savings  for  the  first  eight  years,  but  also  apply  to  a program  con- 
tinuing beyond  eight  years. 

Once  a course  is  formatted  for  CMI  at  remote  sites,  it 
will  also  be  used  in  this  form  at  the  training  center  to  train  students 
whose  base  is  not  outfitted  with  a terminal.  Hence,  COMISAT  must  be 
charged  for  the  additional  training  center  terminals  required  to  teach 
the  course.  The  total  cost  to  COMISAT  is  the  sum  of  the  development 
and  coding,  course  maintenance,  computer  leasing,  training  center  ter- 
minal, and  COMISAT  terminal  costs,  as  summarized  in  Table  II-4.  The 
savings  credited  to  COMISAT  include  the  travel  and  living  costs  for  all 
courses,  student  time  and  incremental  living  savings  for  courses  justi- 
fied only  by  remote  site  savings,  and  training  center  terminal  savings 
for  the  first  17  courses. 

The  net  COMISAT  cost  expression  which  includes  all  of 
the  above  mentioned  costs  and  savings  is: 
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Net  COMISAT  cost 


Development  and  coding 
+ Course  maintenance 
+ Computer  leasing 
+ Training  center  terminals 
+ COMISAT  terminals 

- Travel  and  living 

- Student  time 

- Incremental  living 
+ COMISAT  terminal 

- Travel  and  living 

- Training  center  terminals 


for  courses 
justified  by 
COMISAT  savings 


!for  courses  jus- 
tified by  train- 
ing center  savings 


This  expression  can  be  rewritten  so  that  the  only  unknown  factor  is  the 
number  of  courses  which  should  be  developed.  Maximizing  the  net  cost 
expression  yields  the  optimum  number  of  courses  for  CMI  development.^ 


d.  Determination  of  Additional  COMISAT  Sites 

With  the  number  of  courses  fixed  at  this  new  level,  the 
feasible  sites  are  again  identified  using  a break-even  population  as 
previously  described.  The  iteration  between  courses  and  sites  is  con- 
tinued until  the  net  savings  changes  by  less  than  5 percent  from  one 
cycle  to  the  next.  The  results  obtained  by  this  procedure  are  dis- 
cussed in  Section  B. 


D.  Conclusion 

The  analysis  described  here  provides  a necessary  first  assessment 
of  the  economics  of  the  COMISAT  concept.  The  methodology  developed  for 
the  analysis  can  be  used  as  the  basis  for  more  detailed  analysis  in  the 
future  as  the  COMISAT  concept  and  its  potential  applications  become 
better  defined.  A major  step  in  that  definition  is  confirmation  of  the 
operational  feasibility  of  the  concept  by  demonstration.  Most  impor- 
tant is  the  potential  not  only  for  savings  in  Navy  training,  but 
throughout  the  Department  of  Defense. 


See  Appendix  E. 
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Chapter  III 
RESEARCH  DESIGN 

A.  Introduction 

As  indicated  earlier,  the  COMISAT  project  has  eight  objectives: 

• To  determine  whether  CMI  delivered  to  remote  sites  produces 
the  same  learning  effectiveness  as  CMI  dof s in  the  learning 
center  environment 

• To  determine  whether  the  attitudes  of  students,  trainers,  and 
key  remote  site  personnel  are  supportive  of  CMI  delivered  to 
remote  sites 

• To  determine  whether  CMI  delivered  to  remote  sites  is  as 
economical  as  CMI  in  the  learning  center  environment 

• To  determine  the  personnel  requirements 

• To  determine  the  personnel  training  requirements 

• To  determine  the  organization  and  management  structure  re- 
quirements 

• To  determine  the  remote  site  space  requirements  and  opera- 
tional procedures  for  effective  use  of  a CMI  training  support 
system 

• To  determine  the  equipment,  maintenance,  spare  parts  and 
logistics  requirements 


Achieving  the  first  two  objectives,  concerned  with  learning  effective-  j 
ness  and  the  attitudes  of  students  and  key  personnel,  requires  the  use  j 
of  statistical  methods.  For  the  remaining  six  objectives,  the  infor-  | 
mation  and  data  gathered  during  the  demonstration  will  be  analyzed  on  a i 
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case  study  basis.  The  data  related  to  the  economic  analysis/  for  ex- 
ample, will  be  used  to  check  the  assumptions  made  in  the  preceding 
chapter  on  the  cost  effectiveness  of  an  operational  COMISAT  system. 

The  information  and  data  related  to  personnel,  management,  procedures, 
and  logistics  will  represent  only  one  of  many  situations  to  be  found  at 
Navy  land  bases  and  on  ships.  However,  they  can  be  used  to  infer 
operational  requirements. 

The  following  two  sections  present  descriptions  of  the  statistical 
methods  to  be  used  for  assessing  learning  effectiveness  and  attitudes. 
Succeeding  sections  address  the  data  gathering  and  analysis  to  be  used 
for  each  of  the  six  remaining  objectives. 

B . Statistical  Design  of  the  Assessment  of  Learning  Effectiveness 

1.  General  Statement  of  the  Research  Model 

The  general  approach  for  this  objective  is  an  experimental/ 
control  group  design  where  the  learning  effectiveness  for  a CMI  program 
in  an  operational  training  environment  will  be  systematically  ccxnpared 
to  learning  effectiveness  in  the  training  center  environment.  The 
basic  measure  of  effectiveness  will  be  the  time  required  to  complete 
the  CMI  course.  This  measure  is  used  because  perfect  scores  are  re- 
quired to  pass  from  one  module  to  another  and  complete  the  course. 
Differences  in  learning  are  reflected  only  by  the  difference  in  the 
time  it  takes  students  to  finish. 

A second  concern  with  regard  to  learning  effectiveness  is 
whether  or  not  students  taking  the  CMI  course  at  the  operational  site 
learn  more  about  electricity  and  electronics  than  students  taking  the 
existing  BE/E  correspondence  course.  To  find  out,  volunteers  from  the 
outlying  Stockton  sites  will  be  given  the  correspondence  course.  A 
special  exam  will  then  be  prepared  by  PRC  to  test  the  basic  knowledge 
of  electricity  and  electronics  of  the  students  completing  the  CMI 
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course  and  those  completing  the  correspondence  course.  As  a check  on 
the  validity  of  the  special  exam,  it  will  also  be  given  to  E4  to  E6  RMs 
with  no  formal  training.  If  the  test  is  conceptually  valid,  both  the 
CMI  and  correspondence  groups  should  score  higher  than  those  with  no 
formal  training. 

An  ideal  experimental  design  addressing  these  issues  would 
randomly  assign  subjects  to  the  experimental  and  control  groups  from, 
respectively,  personnel  performing  operational  duties  at  Stockton  and 
trainees  at  a training  center.  This  is  not  possible  at  the  operational 
site  because: 

• The  program  has  to  be  voluntary 

• Only  RMs  are  available  to  participate 

• The  number  of  volunteers  is  small  so  that  all  have  to  be  in- 
cluded in  the  experiment  to  meet  parametric  statistical 
assumptions 

The  fact  that  the  experimental  group  will  consist  of  volunteer  enlisted 
RMs  ranging  from  the  E4  to  E6  paygrades  complicates  the  problem  of  se- 
lecting a control  group.  Since  RMs  are  not  required  to  take  the  BE/E 
course,  none,  or  at  most  one  or  two,  can  be  expected  to  be  taking  it  at 
the  time  of  the  demonstration.  Few  are  even  expected  to  be  found  in 
historical  records  of  the  BE/E  course.  Therefore,  it  is  necessary  to 
select  a basis  for  comparison  other  than  the  same  rating. 

It  was  decided  that  the  basis  for  obtaining  comparable  con- 
trol groups  would  have  to  be  the  paygrade  level  rather  than  the  RM  rat- 
ing. Here  it  was  reasoned  that  the  next  best  thing  to  having  a control 
group  of  the  same  rating  would  be  to  have  the  control  group  consist  of 
personnel  with  roughly  equivalent  experience.  The  basis  for  this  se- 
lection is  that  motivation  and  ability  are  highly  correlated  with  ex- 
perience. This  still  does  not  permit  using  current  trainees  at 
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the  schoolhouse,  because  BE/E  is  em  A- School  course  with  few  trainees 
of  the  higher  paygrade  levels.  However,  there  are  expected  to  be 
enough  historical  records  of  trainees  at  the  E4  to  E6  level  to  obtain  a 
control  group. 

The  experimental  design  for  assessing  learning  effectiveness 
will  therefore  compare  the  time  to  complete  the  CMl  course  by  RMs  at 
Stockton  to  the  performance  of  trainees  with  similar  paygrades  in  the 
historical  records  of  the  Oil  course  at  a training  center.  In  addi- 
tion, the  performance  of  Oil  graduates  from  Stockton  on  a general  elec- 
tricity and  electronics  exam  will  be  compared  to  the  performance  of 
Stockton  personnel  who  complete  the  correspondence  course.  The  design 
is  shown  in  Figure  III-l. 

2.  Research  Hypotheses 

From  the  first  objective  it  is  hypothesized  that: 

The  total  training  hours  required  for  experimental  subjects 
to  complete  the  BE/E  CMI  course  requirements  will  be  less 
than  or  equal  to  the  training  hours  required  for  training 
center  subjects  to  complete  BE/E  course  requirements. 

Statistically  it  is  necessary  to  test  this  hypothesis  by 
first  determining  if  the  following  null  hypothesis,  can  be 
rejected  and  one  of  the  alternatives,  H^^  or  H2,  accepted: 

Hq:  a = t I 

H^:  a >b  ■' 

H2:  a < fc  } 

j 

where  a is  the  mean  time  to  completion  of  the  experimental  group  and  b j 

is  the  mean  time  to  completion  of  the  control  group.  If  H^  is  re-  ! 

O 1 

jected,  then  the  value  of  the  means  will  determine  if  H^  or  H2  is  j 
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to  be  accepted.  Should  not  be  rejected,  the  probability  of 
being  false  will  be  determined. 

It  is  also  hypothesized  that: 

The  CMI  BE/E  training  group  will  perform  better  on  a special 
electricity  and  electronics  examination  than  will  the  corres- 
pondence training  group. 

Stated  symbolically; 

; a = b 

; a > b 

The  null  hypothesis,  assumes  no  effects  due  to  dif- 
ferences in  training  methods,  and  the  alternative  hypothesis, 
predicts  that  the  score,  a , of  the  CMI  BE/E  group  will  be  higher  on  a 
special  electricity  and  electronics  exam  than  the  scores,  b , of  the 
correspondence  group. 

As  a check  on  the  validity  of  the  special  exam,  the  scores  of 
the  CMI  and  correspondence  groups  will  be  compared  to  those  of  a group 
of  E4  to  E6  RMs  with  no  formal  BE/E  training.  Formally  stated,  it  is 
hypothesized  that: 

Both  the  CMI  and  correspondence  course  graduates  will  achieve 
higher  scores  on  the  special  electricity  and  electronics  exam 
than  the  personnel  with  no  formal  BE/E  training. 

The  symbolic  representation  is: 


; a = b = c 

: a > c 

; h > c 
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The  null  hypothesis  assumes  that  neither  of  the  two  training 
^ programs  has  an  e*fect.  The  alternative  hypotheses  and 

^ predict  that  the  scores  a and  t>  of  the  CMI  and  correspondence  groups 

will  be  greater  than  the  scores  e of  the  group  with  no  formal  train- 
' ing-  IE  or  H2  is  not  confirmed,  further  study  would  be  required 

to  determine  if  no  learning  occurred  due  to  CMI  or  correspondence,  or 
if  the  exam  itself  was  poorly  designed. 

. 3.  Definition  of  Research  Variables 

The  independent  variables  are  the  location  of  the  training 
and  the  method  of  training.  The  location  of  training  is  the  more 
important  concern,  since  determining  the  differences  in  learning 
effectiveness,  if  any,  between  the  CMI  course  at  the  remote  site  and  at 
a training  center  is  the  major  objective  of  the  COMISAT  project.  The 
methods  of  training  investigated  in  this  project  are  the  CMI  and  cor- 
respondence  courses. 

The  dependent  variable  for  determining  the  effect  of  location 
on  CMI  performance  is  the  total  amount  of  time  required  to  reach  * 

criterion  on  Modules  1-14  of  the  BE/E  course.  This  includes  all  of  the 
students'  time  spent  studying,  doing  laboratory  experiments,  and  taking 
performance  and  knowledge  tests  for  each  module.  The  variable  is  ex- 
pressed in  total  hours  spent  for  the  course. 

The  dependent  variable  for  determining  the  difference  in  ef- 
fectiveness between  CMI  and  the  correspondence  course  training  methods 
is  the  score  on  a specially  designed  test  which  measures  knowledge  of 
electricity  and  electronics  materials  contained  in  both  the  CMI  BE/E 
course  (Reference  10),  and  the  correspondence  course  manual.  Chapters 
1-15  (Reference  11).  The  test  will  draw  from  material  common  to  both 
the  CMI  and  correspondence  materials. 


J 
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The  covariate  or  intervening  variables  are  used  to  adjust  the 
* I values  of  the  dependent  variables  to  account  for  initial  differences 

V'  , between  the  people  whose  performance  is  being  measured.  For  example, 

^ the  CMI  students  at  Stockton  might  have  consistently  different  Basic 

Test  Battery  (BTB)  scores  them  the  training  center  control  group.  The 
BTB  is  given  to  each  enlistee,  and  is  a standard  Navy  aptitude  measure 
, . used  in  part  in  assigning  enlistees  to  ratings.  If  the  BTBs  are  dif- 

' ferent,  then  a difference  in  CMI  performance  can  be  expected.  The 

, I covariance  analysis  statistically  matches  groups  so  that  the  differ- 

ences  due  to  location  of  the  training  can  be  assessed. 

The  BTB  scores  will  be  used  as  the  covariate  in  comparing 
, learning  effectiveness  at  the  site  to  that  at  the  training  center.  The 

i scores  have  been  shown  to  be  negatively  correlated  with  the  measure  of 

t 

performance  in  the  course;  i.e.,  people  with  higher  BTB  scores  take 
less  time  to  complete  the  course.  Because  of  significant  correlations, 
BTB  scores  are  presently  used  as  the  basis  for  predicting  the  comple- 
tion time  for  each  CMI  student.  The  predictions  are  used  by  the  LS  in 
monitoring  student  progress. 

Both  BTB  and  diagnostic  reading  scores  will  be  used  as  covar- 
iates in  comparing  student  performance  on  CMI  to  correspondence  student 
performance.  Again,  previous  evidence  shows  a correlation  between  this 
ability  and  success  in  training.  This  correlation  was  particularly 
significant  in  electricity  and  electronics  training. 


4.  Statistical  Models 

One-way  analysis  of  covariance  statistics  will  be  used  to 
test  these  hypotheses.  Symbolically,  the  model  is  as  follows: 


1 

( 
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where  observation  on  the  dependent  measure,  e.g., 

total  time  to  complete  BE/E  course,  under  the  ith  treat- 
ment, e.g.,  experimental  or  control  site; 


= the  covariate  variable(s),  e.g.,  Basic  Test  Battery 
^ score,  corresponding  to 

X. . . the  mean  of  x_  scores; 

= the  overall  mean; 


r . 
1 

0 


Ci: 


the  effect  of  the  ith  treatment; 

a linear  regression  coefficient  showing  dependency  of 
Yij  on  x.j; 

the  random  error  component; 


The  model  assumes  that: 

• Errors  are  normal  and  independently  distributed, 

• 0=0  and  the  relationship  between  y.  . and  x - • is 
linear 

• Regression  coefficients  for  treatments  are  identical 

• Treatment  effects  sum  to  zero 

• The  covariate  is  not  affected  by  the  treatment 


The  F statistic  is  used  to  test  H . t - 0 

O'  i 


(SS'£  - SSE)/(a  - 1) 

F = 

° SSg/  a(n-l)  - 1 


where 


"^E 

SS’, 


(SS’g  . ss 


E) 


a 

n 


error  sum  of  squares; 

reduced  sum  of  squares  for  error; 

the  reduction  in  sums  of  squares  in  order 
to  test  the  hypothesis  of  no  treatment 
effects; 

the  number  of  treatment  (experimental)  groups; 
the  number  of  subjects  per  group. 
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The  null  hypothesis  will  be  rejected  if  . 

The  alpha  level  or  level  of  confidence  for  statistical  tests  is  set  at 
p < .05 . 


It  should  be  noted  that  for  the  second  hypothesis,  i.e.,  de- 
termining the  effects  of  CMI  and  correspondence  training  methods  on 
learning  electricity  and  electronics  material,  it  may  not  be  possible 
to  use  the  covariance  analysis  because  of  the  small  number  of  partici- 
pants anticipated  in  the  correspondence  course.  Should  the  sample  size 
fall  below  20,  it  may  be  necessary  to  use  an  equivalent  nonparametr ic 
statistic.  The  appropriate  model  for  this  situation  is  the  Kruskal- 
Wallis  one-way  analysis  of  variance  by  ranks  statistic.  Stated 
symbolically; 


K = 


12 


N(N  + 1) 


= 1 


R. 

1 


n . 
3 


3(R  + 1) 


where  k = number  of  samples 


n . = 
3 

N = 

R.  = 
3 


number  of  cases  in  jth  sample 

2^:  ^ the  number  of  cases  in  all  samples  combined 
sum  of  ranks  in  jth  sample  (column) 


The  H statistic  is  used  to  test  the  hypothesis  and,  with  a sample 
greater  than  five,  is  distributed  approximately  as  chi  square  with 
k - 1 degrees  of  freedom. 


5.  Data  Collection,  Organization  and  Analysis 
a.  Input 

At  the  conclusion  of  the  demonstration  all  data  will  be 
transferred  from  data  tally  sheets  to  punched  cards  for  the  analysis. 
Appendix  H contains  a sample  data  tally  sheet  and  shows  the  data  format 
for  the  punched  cards. 
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The  BTB  data  will  be  recorded  on  data  tally  sheets  by 
PRC  staff  with  the  assistance  of  the  ESO  during  the  week  of  September 
12.  The  reading  ability  scores  will  be  recorded  on  tally  sheets  by 
October  21.  The  last  data  input  will  be  the  CMI  course  performance, 
correspondence  course  performance,  and  performance  on  the  electricity 
and  electronics  special  exam.  These  will  have  been  collected  by  March 
10,  1078. 

An  added  requirement  for  the  Kruskal-Wallis  analysis  is 
that  the  data  must  be  arranged  as  n cases  of  two  variables,  where  n is 
the  number  of  subjects.  One  variable  is  the  test  score;  the  other  is 
an  identifier  which  indicates  whether  the  subject  is  in  the  CMI, 
correspondence,  or  no-formal-training  group.  The  input  format  for 
these  data  is  also  shown  in  the  sample  punched  card  in  Appendix  H. 

b.  Computational  Steps 

The  computational  steps  for  the  covariance  analysis  are 
quite  lengthy.  The  reader  can  find  a discussion  of  the  procedures  for 
one-way  analysis  of  covariance  in  Reference  12.  For  the  Kruskal-Wallis 
analysis  computation  the  procedure  is  far  less  complicated.  The  reader 
is  again  referred  to  Reference  12  for  the  specific  computational 
procedures. 

c . Output  and  Interpretation  of  Results 

The  computer  output  for  the  covariance  analysis  includes 
a number  of  important  items.  These  are; 

• Regression  coefficients,  their  standard  errors  and  t values 

• Group  means,  adjusted  group  means,  and  standard  errors  of  ad- 
justed group  means 

• Analysis  of  variance  table  with  F test  for  equality  of  adjus- 
ted group  means,  zero  slope,  equality  of  slope,  and  the  tail 
area  probability 


51 


TAEG  Report  No,  49 


• Pair  wise  t test  for  adjusted  group  means  and  t test  for  con- 
trasts of  adjusted  group  means 

• Regression  coefficients  for  each  covariate  in  each  group 

Proper  interpretation  of  these  results  requires  that  a 
sequence  of  steps  be  followed: 

• The  regression  coefficients  should  be  the  same  for  treatments 
in  each  of  the  CMI , correspondence,  and,  where  appropriate, 
no-formal-training  groups.  If  they  are  not,  the  relationship 
between  the  selected  covariate  and  the  dependent  variable  is 
different  for  each  group  being  measured.  Another  covariate 
would  have  to  be  found  which  does  not  have  a different  rela- 
tionship for  each  group,  data  obtained  for  the  new  covariate, 
and  the  analysis  repeated  until  a satisfactory  covariate  is 
identi f ied. 

• The  F test  of  the  equality  of  slopes  should  be  checked  to 
confirm  that  the  regression  lines  for  the  three  groups  have 
the  same  slope.  If  the  slopes  are  significantly  different, 
the  covariance  analysis  is  not  appropriate  and  analysis  of 
variance  would  be  substituted. 

• For  the  first  hypothesis  concerning  course  performance  the  F 
test  for  the  equality  of  adjusted  group  mean  is  the  most  im- 
portant result  from  the  output.  Provided  the  previous  as- 
sumptions have  been  met,  a significant  F ratio  will  indicate 
that  the  two  training  environments  produced  course  perfor- 
mance results  that  were  significantly  different  from  each 
other.  The  more  effective  training  environment  will  have  a 
lower  adjusted  group  mean  indicating  less  time  required  to 
reach  criterion. 

• If  covariance  is  used  to  determine  the  effects  of  different 
training  methods,  then  a significant  F ratio  would  require 
checking  t ratios  since  there  would  be  three,  rather  than 
two,  groups  involved.  Checking  the  t ratios  becomes  neces- 
sary in  this  case  because  a significant  F ratio  does  not  in- 
dicate which  group  means  are  significantly  different  from 
each  other,  it  only  indicates  that  there  is  a difference. 

The  t test  will  show  the  ordering  of  the  three  training 
methods  in  terms  of  test  performance  and  the  level  of  signif- 
icance. The  level  of  significance  of  the  difference  in  per- 
formance between  the  CMI  and  correspondence  groups  should 
also  be  assessed  by  conducting  an  F test  of  these  two  groups 
without  considering  the  control  group. 
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The  output  for  the  Kruskal-Wallis  test  statistic  is  less 
complicated  and  includes  the  items  listed  below: 

• Test  statistic  H 

• Level  of  significance  or  probability  of  occurrence 

Samples  of  the  printout  for  each  of  these  analyses  are 
provided  in  Reference  12. 

If  the  Kruskal-Wallis  analysis  is  used  for  the  second 
hypothesis  test  of  the  relative  effectiveness  of  training  methods,  the 
H statistic  is  interpreted  in  the  same  manner  as  the  F statistic  but  by 
using  the  chi  square  distribution  rather  than  the  F distribution.  A 
significant  H will  be  interpreted  as  confirmation  of  the  alternative 
hypothesis,  i.e.,  that  CMI  is  better  than  correspondence  or  no  formal 
training,  if  the  group  mean  differences  are  in  the  predicted  direc- 
tion. A significant  H test  with  the  'no  formal  training'  group  j_n  the 
analysis  would  lead  to  a recommendation  that  additional  analysis  be 
conducted  without  the  control  group.  This  analysis  would  provide  a di- 
rect assessment  relative  to  a treatment  effect  due  to  CMI  training  ver- 
sus correspondence  training. 

C.  Definitions  and  Procedures  for  the  Attitude  Objective 

1.  General  Statement  of  the  Research  Model 

For  this  objective  attitude  measures  will  be  compared  between 
operational  site  students  and  a comparable  group  of  individuals  taking 
a CMI  course  at  training  center  facilities.  A comparison  of  attitudes 
will  also  be  made  between  the  LS  at  the  site  and  LSs  at  the  training 
centers.  A third  concern  is  to  assess  the  attitudes  of  key  Stockton 
operational  personnel  towards  CMI. 

Semantic  differential  scales  have  been  designed  to  measure 
attitudes  toward  a number  of  CMI  dimensions.  These  include  CMI  as  a 
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concept,  course  material,  the  LS  as  viewed  by  the  students,  the  stu- 
dents as  viewed  by  the  LS,  testing  procedures,  and  test  feedback  in 
terms  of  content  and  response  time.  A pretest-posttest  approach  will 
be  followed  in  collecting  the  data. 

Statistical  methods  can  be  used  in  comparing  experimental  and 
control  student  groups.  For  the  LS  and  key  personnel  at  Stockton,  how- 
ever, comparative  analysis  of  attitudes  will  not  be  possible.  In  the 
case  of  the  LS,  it  will  be  possible  to  test  several  LSs  at  the  training 
center.  This  information  can  only  be  used  as  a guide  in  searching  for 
conditions  at  Stockton  that  strongly  affect  the  LS's  attitudes  towards 
the  delivery  of  CWI  at  a remote  site.  No  comparison  will  be  possible 
for  the  attitudes  of  key  Stockton  personnel.  However,  the  attitude 
test  results  will  be  useful  in  determining  the  conditions  most  affect- 
ing their  attitudes. 

The  following  subsections  describe  the  analysis  of  student 
attitude  data.  The  final  subsection  briefly  discusses  criteria  for  as- 
sessing LS  and  key  personnel  attitudes. 

a . Demonstration  Groups 

The  experimental  student  group  for  the  attitude  study 
will  be  the  CMI  students  at  Stockton.  Since  the  control  group  for  the 
learning  effectiveness  analysis  is  drawn  from  historical  records,  it 
cannot  be  used  in  the  attitude  study.  A separate  control  group  is 
needed . 

The  objective  is  to  determine  attitudes  toward  CMI  as  a 
delivery  system  rather  than  the  specific  course.  Therefore,  although 
it  is  possible  to  find  subjects  in  any  CMI  course  at  any  training  cen- 
ter, the  important  factor  is  that  they  be  similar  to  the  experimental 
subjects  with  respect  to  length  and  level  of  service  experience.  That 
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is  best  represented  by  pay  grade.  The  reasoning  behind  this  choice  is 
that  people  with  the  same  length  of  service  experience  would  be  more 
likely  to  have  similar  motivation  and  reactions  to  learning  conditions. 

In  order  to  adjust  for  differences  not  eliminated  by  the 
use  of  equivalent  pay  grades  in  the  control  group,  covariance  analysis 
will  be  used  in  evaluating  attitudes.  Based  on  previous  research  with 
semantic  differential  instruments,  pretest  measures  have  been  shown  to 
be  a good  means  of  adjusting  posttest  measures  to  account  for  initial 
differences  between  the  experimental  and  control  groups  (Reference  13) . 

Although  any  training  center  is  a candidate  for  provi- 
ding attitude  control  groups,  it  is  expected  that  Memphis  will  be  the 
choice.  The  only  factor  that  can  change  this  is  the  possibility  that 
not  enough  students  at  the  required  pay  grades  will  be  found  at  the  one 
school . 

b.  Validation  Groups 

Prior  to  demonstration  data  collection,  a conceptual  and 
statistical  validation  of  the  student  attitude  instrument  is  neces- 
sary. A sample  of  at  least  110  subjects  will  be  needed.  The  BE/E 
schools  at  Great  Lakes  and  Memphis  appear  to  have  the  most  represen- 
tative sample  of  BE/E  CMI  students  and  will  therefore  be  used  for  the 
validation  study. 

The  validation  study  will  use  factor  analysis  to  reduce 
the  number  of  variables  used  in  testing  attitudes  toward  each  CMI  com- 
ponent. The  resulting  version  of  the  test  will  be  more  accurate  and 
conceptually  valid.  Figure  III-2  shows  the  design  approach,  including 
validation,  for  the  attitude  objective. 

2.  Research  Hypothesis 

It  is  hypothesized  that: 
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Figure  III  2.  Research  Design  For  CM!  A ttitude  A nalyses 


Attitudes  toward  CMI  cxsncepts  for  key  site  personnel  and 
trainees  as  measured  by  semantic  differential  scores  will  be 
equal  to  or  more  supportive  than  attitudes  of  trainees. 

Stated  symbolically; 


•,  o > b 

^2  * ci  ^ b 

where  the  null  hypothesis,  assumes  no  attitude  effect  due  to  the 
different  training  sites,  and  the  alternative  hypothesis,  pre- 
dicts that  training  in  the  operational  context  will  produce  attitudes 
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toward  the  CMI  concept  more  supportive  than 
center.  is  rejected,  the  mean  scores 

termine  if  training  at  the  operational  site 
the  schoolhouse  or  vice  versa. 

3.  Definition  of  Variables 

The  training  site  is  the  major  independent  variable  for  atti- 
tude measures.  The  experimental  site  has  already  been  defined  as  the 
operational  training  site  at  NAVCOMMSTA  Stockton.  The  control  site(s) 
will  be  one  or  more  training  center  facilities  with  CMI. 


at  the  traditional  training 

will  be  examined  to  de- 
ls more  desirable  than  at 


The  dependent  variables  will  be  attitudes  toward  the  CMI  com- 
ponents addressed  by  the  semantic  differential  scales.  The  semantic 
differentials  were  developed  with  bipolar  adjectives  considered  rele- 
vant to  that  concept.  As  an  example,  the  bipolar  adjectives  considered 
relevant  to  the  CMI  concept  are  shown  in  Figure  III-3.  The  complete 
instrument  addressing  this  and  the  other  components  is  presented  in 
Appendix  I. 


As  indicated  earlier,  pretest  attitude  scores  will  serve  as  ] 

I 

the  covariate  for  each  analysis.  j 

I 


4 . Instrument  Validation. 

As  indicated  above,  the  statistical  validation  of  the  atti- 
tude instrument  requires  appropriate  factor  analytic  techniques.  The 
basic  statistical  model  for  this  validation  stated  symbolically  is: 


z , 
1 


+ a F 
jn  n 


where 


a variable  j in  standardized  form,  described 

linearly  in  terms  of  F and  a- 

n “jn 
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Computer  Managed  Instruction  ICMI) 
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Figure  111-3.  Semantic  Differential  Instrument:  CM!  Concept 


= new  uncorrelated  components  I 

= standardized  regression  coefficients 

Factor  analysis  is  used  to  insure  measurement  reliability  and 
conceptual  validity  by  removing  unnecessary  measurement  overlap.  Es- 
sentially, the  procedure  will  allow  us  to  determine  which  adjectives  in 
the  semantic  differentials  are  intercor related.  The  major  objective  of  , 

the  analysis  is  to  arrive  at  simplified  subjective  measures  for  each  of 
the  concepts  in  the  study.  In  the  present  case,  these  measures  derive 
from  a subject's  response  to  the  semantic  differential  instrument  (see 
Figure  III-3).  The  result  of  the  factor  analysis  is  a smaller  number 
of  bipolar  adjectives  being  abstracted  from  the  larger  instrument.  The 
adjectives  which  are  abstracted  from  the  larger  set  represent  the  vari- 
ous ways  that  respondents  interpret  each  concept.  In  response  to  the 
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survey,  those  adjectives  which  are  correlated  with  each  other  or  are 
used  in  a similiar  fashion  by  subjects  are  said  to  represent  a factor. 

A factor  is  a theoretical  construct  which  is  assumed  to  characterize 
one  dimension  of  a survey  or  test.  In  education,  for  example,  verbal 
ability  is  thought  to  be  one  of  several  factors  underlying  intelligence. 

As  shown  in  Figure  III-4,  the  data  indicate  that  one  of  the 
factors  underlying  the  CMI  concept  has  five  bipolar  adjective  mea- 
sures. These  adjectives  all  have  relatively  high  factor  loadings 
(between  .69  and  .84)  under  Factor  I and  relatively  low  loadings  under 
each  of  the  other  factors  in  the  matrix.  All  five  adjectives  would 
serve  as  a valid  measure  of  Factor  I.  Since  these  adjectives  represent 
attitudes  as  opposed  to  other  factors  in  the  matrix.  Factor  I has  been 
labeled  "attitude."  Theoretically,  scores  from  any  one  of  the  five 
could  serve  as  the  attitude  measure.  However,  greater  measurement  re- 
liability is  obtained  by  summing  scores  from  all  five  adjectives. 

Based  on  adjectives  with  high  factor  loadings,  our  hypothetical  example 
yields  three  other  factors  which  we  have  arbitrarily  labeled: 

"adequacy  of  the  system,”  Factor  II;  "system  meaningfulness,"  Factor 
III;  and  "system  appeal,"  Factor  IV.  Since  our  primary  interest  in 
this  study  is  the  measurement  of  attitudes  toward  the  CMI  concept  as 
included  in  Factor  I,  only  those  five  adjectives  would  be  used  in  the 
analysis.  The  revised  version  of  the  CMI  concept  test  based  on  the 
factor  loadings  is  shown  in  Figure  III-5. 

For  the  pilot  validation  the  principle  factors  technique  fol- 
lowed by  an  orthogonal  rotation  to  simple  structure  with  the  varimax 
criterion  will  be  used.  An  eigenvalue  of  1.0  or  greater  will  be  used 
as  criterion  for  terminating  factor  extraction  in  the  rotation  proce- 
dure (see  Reference  14) . In  the  present  case,  the  factor  analysis  is 
performed  on  the  n x n correlation  matrix  formed  from  each  set  of  se- 
mantic differential  scales.  Essentially,  the  factor  analysis  identi- 
fies clusters  of  adjectives  from  the  correlation  matrix  which  are 
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Factor 

Variable 

1 

II 

III 

IV 

Like-dislike 

.81* 

.11 

.09 

.13 

valuable-worthless 

.78* 

.05 

.19 

.03 

adequate-inadequate 

,12 

.68 

.28 

.04 

meaningless-meaningful 

.07 

.21 

.65 

.06 

interesting-boring 

.02 

,14 

.09 

.79 

frustrating-motivating 

.01 

.27 

.04 

.29 

good-bad 

.84* 

.10 

.01 

.16 

unorganized-organized 

.14 

.23 

.82 

.14 

disapprove-approve 

.69* 

.23 

.08 

.13 

useless-useful 

.73* 

.18 

,11 

.01 

*These  variables  loaded  highly  under  one  factor  (Factor  I)  and  were 
relatively  pure;  i.e  , had  relatively  low  loading  on  all  other  factors.  They 
would  therefore  be  used  as  the  attitude  measure  for  the  CMI  concept. 

Figure  111-4.  Hypothetical  Rotated  Factor  Matrix  for  the 
Computer  Managed  Instruction  Semantic 
Differential  Instrument 


dislike 

worthless 

bad 

approve 

useful 


Figure  III-5.  Semantic  Differential  Test:  CMI  Concept 
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intercorrelated  and  bear  a relation  to  one  or  more  factors.  This 
analysis  results  in  a factor  matrix  containing  factor  loadings.  Factor 
loadings  are  basically  an  indication  of  the  degree  of  relation  between 
a bipolar  adjective  measure  and  the  factors.  From  the  raw  factor  ma- 
trix an  orthogonal  rotation  is  performed  so  that  the  clusters  of  adjec- 
tives can  be  arbitrarily  placed  in  the  most  meaningful  psychological 
array.  This  rotation  yields  the  factor  matrix  depicted  in  Figure 
III-4.  In  this  exaunple  the  eignevalue  criterion  reduced  the  number  of 
meaningful  factors  in  the  rotated  matrix  to  four.  Other  factors  are 
dropped  from  the  matrix  because  they  account  for  virtually  none  of  the 
variance  in  the  survey. 

The  criterion  for  acceptance  of  an  item  in  the  factor  struc- 
ture will  be  set  at  a factor  loading  of  .60  or  better  on  one  factor  and 
.30  or  less  on  any  other  factor  in  the  matrix.  While  not  anticipated, 
some  modification  of  this  criterion  may  be  necessary  depending  on  the 
outcome  of  the  rotated  factor  matrix.  (Some  researchers  use  less 
stringent  criteria  for  placing  a scale  into  a given  factor  in  the  ma- 
trix, such  as  .50  and  .40  instead  of  .60  and  .30  respectively.) 

Once  the  instrument  has  been  validated,  a one-way  analysis  of 
covariance  will  be  used  for  the  demonstration  data.  The  covariance 
model  for  the  attitude  analysis  is  the  same  as  that  for  the  course  per- 
formance analyses. 

References  14  and  15  are  recommended  for  an  in-depth  discus- 
sion of  factor  analysis  theory  and  application. 

5.  Data  Collection,  Organisation,  and  Analysis  of  Student 

Attitudes 

a.  Input 

At  the  conclusion  of  the  demonstration  all  data  will  be 
transferred  from  the  data  tally  sheets  to  punched  cards  for  analysis  as 
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shown  in  Appendix  H.  Pretest  attitude  data  at  experimental  and  control 
sites  will  be  collected  and  tabulated  by  PRC  staff  with  the  assistance 
of  the  ESO  during  the  week  of  August  18.  In  the  case  of  the  control 
subjects,  pretest  data  collection  may  be  at  more  than  one  site  as  de- 
signated. This  data  collection  will  be  done  by  PRC  staff  with  assis- 
tance from  designated  personnel  at  the  control  group  sites.  Posttest 
attitudes  will  be  collected  and  tabulated  by  PRC  staff  with  assistance 
from  the  ESO  and  designated  assistants  at  the  conclusion  of  each 
student's  course.  It  is  estimated  that  the  last  student  will  finish 
during  the  last  week  of  February. 


b.  Computations  and  Results 

The  covariance  analysis  will  be  similar  in  form  to  that 
described  for  evaluation  of  course  performance  data.  A nonsignificant 
F ratio,  or  a significant  F ratio  in  favor  of  experimental  group  atti- 
tudes, will  confirm  the  hypothesis  that  student  attitudes  at  the  remote 
site  are  supportive  of  Oil. 


6.  LS  and  Key  Personnel  Attitudes 

The  student-validated  instrument  will  be  used  to  assess  the 
attitudes  of  the  LS  and  key  Stockton  personnel.  As  indicated  earlier, 
the  LS's  attitudes  towards  CMI  delivery  at  remote  sites  will  be  com- 
pared to  the  attitudes  of  LSs  at  the  training  centers.  The  LSs  of  the 
control  group  students  will  be  given  the  pretest-posttest  combination. 
This  will  probably  involve  no  more  than  six  to  10  subjects. 

The  attitudes  expressed  by  the  Stockton  LS  will  be  compared 
to  the  mean  scores  of  the  training  center  LSs.  Areas  of  differences 
will  b^  explored  to  find  out  what  conditions  most  affect  the  LS's  atti- 
tude. This  information  will  be  supplemented  by  interview  data  obtained 
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in  support  of  the  personnel,  personnel  training,  facilities,  proce- 
dures, and  logistics  objectives  of  the  demonstration.  The  interviews 
are  described  in  later  sections  of  this  chapter. 

Key  personnel  attitudes  will  also  have  to  be  examined  for  in- 
dications of  major  concerns.  In  this  case  there  will  be  no  comparative 
information  to  guide  the  assessment  (as  there  is  with  the  LS) . The  re- 
lative measures  from  the  tests,  and  the  change  in  those  measures,  will 
provide  useful  information.  Additional  understanding  of  areas  of  sa- 
tisfaction and  dissatisfaction  w,!!  be  obtained  frcxn  the  interview  res- 
ponses gathered  for  the  other  objectives. 

For  both  the  LS  at  Stockton  and  the  key  Stockton  personnel,  a 
score  of  4.00  or  higher  on  any  aspect  of  the  CMI  system  will  be  re- 
garded as  favorable  since  it  is  above  the  hypothetical  mean.  That 
finding  would  confirm  the  hypothesis  that  key  personnel  will  be  sup- 
portive of  CMI  delivered  to  remote  sites.  In  addition,  the  assessment 
of  major  factors  influencing  attitudes  can  lead  to  recommendations  for 
an  improved  delivery  system  design. 

D.  Economic  Evaluation 

1.  Data  Gathering  Objectives 

The  purpose  of  the  economic  evaluation  is  to  obtain  improved 
estimates  on  operational  costs  for  the  COMISAT  concept.  Those  esti- 
mates would  permit  updating  the  analysis  of  Chapter  II  to  obtain  a more 
accurate  projection  of  net  savings  to  the  Navy  attainable  through  a 
fully  operational  remote-site  (ZMI  network.  The  specific  data  gathering 
objectives  of  this  evaluation  are  as  follows; 

• To  determine  all  costs  of  procuring,  installing,  maintaining, 
and  operating  the  communication  system  required  for  the  CMI 
demonstration.  This  will  require  data  on  the  reliability  and 
maintenance  of  the  terminals,  particularly  the  Opscan,  and  on 
the  reliability  of  the  communication  link. 
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• To  determine  the  equipment  and  materials  (including 
expendables)  required  to  keep  the  CMI  course  in  operation. 

• To  determine  the  amount  of  additional  personnel  hours  ex- 
pended on  activities  connected  with  the  CMI  demonstration  ob- 
jectives by; 

Each  student 
The  LS 
The  ETs 

Watch  supervisors 
The  ESO 

Division  Training  Chiefs/Petty  Officers  (DTC/POs) 

Division  Officers 

Communications  Officer 

CMI  advisor 

Other  staff 

• To  determine  the  equivalent  costs  of  operating  the  corres- 
pondence course: 

Student  hours  required 
Cost  of  materials 
Cost  of  mail 

Remote  site  personnel  hours  required 


This  information,  along  with  the  assessment  of  correspondence 
course  effectiveness,  will  be  used  to  compare  CMI  and  correspondence  as 
a means  of  reducing  Navy  training  costs. 


2.  Data  Gathering  Procedures 

a.  Communication  System  Cost 

At  the  end  of  the  demonstration,  MIISA  Memphis  will  pro- 
vide PRC  with  a complete  cost  breakdown  of  the  communication  system. 

The  required  data  are  as  follows; 

• Lease,  including  installation  and  maintenance,  of  two 
Opscan/Terminet  clusters 

• Lease,  including  installation  and  maintenance,  of  a commer- 
cial dedicated  line  from  Stockton  to  San  Diego,  including  a 
Bell  202-T  modem  at  each  end  of  the  line 
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• Lease,  including  installation  and  maintenance,  of  an  RP6352 
asynchronous  line  modulator  to  connect  the  phone  line  to  the 
CWI  concentrator  at  San  Diego 


These  data  will  be  collected  and  mailed  to  PRC  in 
McLean,  arriving  by  March  10,  1978. 


As  part  of  the  assessment  of  communication  system  costs, 
the  data  collection  system  will  measure  the  system's  reliability  and 
maintenance  requirements  in  the  following  way; 

• When  either  the  Opscan  or  Terminet  malfunctions,  the  LS  or  ET 
enters  into  the  maintenance  log  (see  Appendix  J) : 

— Identification  of  the  equipment 
— The  time  at  which  the  failure  was  first  noticed 
— The  time  the  contractor  is  called 
— The  nature  of  the  malfunction 

On  arrival,  the  maintenance  technician  enters  the  time.  When 
the  work  is  complete  and  the  equipment  back  in  operation  the 
time  is  entered  in  the  log  and  the  nature  of  the  problem  and 
the  correction  are  described. 

• Down  time  resulting  from  problems  with  the  communication 
link,  the  CMI  concentrator  in  San  Diego,  or  the  CMI  computer 
in  Memphis,  will  be  recorded  by  the  LS  or  ET.  If  not  immedi- 
ately known,  the  LS  or  ET  will  determine  the  cause  by  con- 
tacting MIISA  San  Diego  or  Memphis  and  recording  it  in  the 
maintenance  log. 


On  the  first  day  of  each  month  the  LS  will  send  a copy 
of  the  maintenance  log  to  PRC  in  McLean,  Virginia. 

b.  Personnel  Resources 

(1)  Student  Course  Time 

Each  student  in  both  the  CMI  course  as  well  as  the 
correspondence  course  will  be  issued  a time  card  as  shown  in  Appendix 
I.  A complete  chronological  history  of  time  spent  in  study  (on  and  off 
watch),  lab,  performance  test,  and  knowledge  test  will  be  kept.  The 
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CMI  students  will  have  their  time  stored  in  the  computer.  For  them  the 
card  is  a reference  from  which  corrections  can  be  made  if  there  are  er- 
rors in  the  computer  entries.  The  LS  will  make  any  corrections  that 
are  needed.  Final  data  on  the  total  course  time  will  be  provided  to 
PRC  by  MIISA  by  February  28. 

(2)  LS  Time 

Time  spent  by  the  LS  and  the  type  of  activity  will 
be  recorded  in  the  log  described  in  Appendix  J.  The  data  to  be  re- 
corded each  day  are: 

• Date 

• Hours  devoted  to  counseling  the  students  on  course  content 
problems 

• Hours  devoted  to  administrative  functions,  e.g. , maintaining 
student  files,  answering  procedural  questions,  and  coordi- 
nating activities  with  Fleet  Center  operations 

(3)  ET  Time 

When  the  ETs  act  as  resource  persons  in  the  absence 
of  the  LS,  their  time  is  to  be  kept  on  a log  identical  to  the  LS  form. 
The  entries  would  be  made  in  the  Substantive  Activities  section.  Ad- 
ministrative activities  will  be  entered  separately. 

The  ET  lab  equipment  maintenance  time  will  be  re- 
corded in  the  same  manner  that  the  log  is  kept  for  Opscan/Terminet 
maintenance. 

(4)  Other  Personnel 

ESO,  Communications  Officer,  Division  Officers, 
DTC/POs  and  CMI  advisor  will  each  maintain  a key  personnel  log.  When 
staff  of  the  Fleet  Center  perform  duties  on  the  demonstration,  the  Di- 
vision Officers  will  indicate  in  their  logs  the  type  and  duration  of 
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the  activity,  and  the  position  of  the  person  doing  it.  If  an  indivi- 
dual not  assigned  a log  is  found  to  contribute  a significant  amount  of 
time  to  the  demonstration,  that  person  will  be  assigned  a log  of  his  or 
her  own. 

c.  Equipment  and  Materials 

The  concern  here  is  to  determine  the  amount  and  cost  of 
lab  equipment,  microfiche  readers,  microfiche  test  cards,  texts,  answer 
sheets,  administrative  forms  (PI,  P2,  P3),  Terminet  paper,  and  expend- 
ables required  per  student.  The  BE/E  Training  Program  Coordinator 
(TPC)  at  CNTECHTRA  will  provide  PRC  at  McLean  with  an  itemized  cost  of 
each  shipment. 

It  will  be  important  to  know  if  the  shipments  adequately 
Support  the  operation.  Therefore,  the  LS  and  ETs  will  note  in  their 
logs  anv  instances  where  equipment  or  materials  are  not  available  to  a 
student . 


d.  Correspondence  Course  Costs 


The  following  data  gathering  will  take  place  during  the 


demonstration: 


The  correspondence  course  student  time  data  will  be  maintain- 
ed by  the  ESO  at  Stockton.  Each  Friday  each  student  is  to 
give  his  or  her  weekly  card  to  the  DTC/PO  to  be  sent  by  Navy 
mail  back  to  the  ESO  at  headquarters.  Once  a week,  the  ESO 
will  send  a summary  of  student  progress  to  PRC  at  McLean  and 
give  a copy  to  PRC  staff  on  site,  if  any,  showing 
Each  student's  name 
Lessons  completed 
Lesson  being  worked 
Total  time  in  the  course  to  date 
The  format  is  shown  in  Appendix  J. 

The  ESO's  time  spent  on  correspondence  will  be  recorded  sepa- 
rately in  his  log. 


67 


r 


TAEG  Report  No.  49 

The  DTC/POs  at  the  outlying  sites  will  maintain  a key  person- 
nel log,  entering  their  time  under  the  Administrative  Activ- 
ity heading.  The  ESO  is  to  receive  the  logs  by  base  mail 
each  Monday,  and  copies  are  to  be  sent  to  PRC  at  McLean  along 
with  the  CMI  personnel  logs. 

Estimates  will  be  sought,  based  on  correspondence  course  ex- 
perience, for  the  cost  of  texts,  tests,  answer  sheets,  and 
mailing  materials  back  and  forth  between  the  site  and  the 
correspondence  center.  The  estimates  will  be  obtained  from 
the  Naval  Education  and  Training  Program  Development  Center, 
Pensacola,  Florida. 

Analysis  and  Results 

The  data  obtained  for  the  economic  evaluation  will  be  used  in 
recalculating  the  cost  impact  of  the  COMISAT  project  with  the  procedure 
described  in  Chapter  II.  The  communication  line  and  modem  costs  for 
the  commercial  leased  line  will  serve  only  as  a guide  to  cost  and  reli- 
ability problems.  If  COMISAT  goes  operational  it  will  use  AUTODIN  II 
and  therefore  would  not  incur  commercial  modem  equipment  and  leased 
line  costs.  All  the  other  data  on  terminals,  equipment,  materials,  and 
personnel  will  be  used  to  determine  potential  costs  and  savings  of  an 
operational  system. 

E.  Personnel  Requirements 

This  objective  is  concerned  with  determining  the  tasks  that  have 
to  be  performed  by  onsite  personnel  and  the  level  of  effort  required  in 
order  to  support  an  effective  CMI  program.  The  personnel  of  interest 
are  the  LS,  ETs,  DTC/POs,  ESO,  Division  Officers,  and  the  Communi- 
cations Officer. 

The  information  and  data  needed  to  achieve  this  objective  come 
from  three  sources: 

• PRC  staff  will  observe  operations  continuously  over  the  first 
two  weeks  of  the  demonstration,  and  then  for  one  week  inter- 
vals at  the  beginning  of  the  second  month  of  the  demonstra- 
tion, and  once  a month  thereafter. 
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• PRC  staff  will  interview  students  and  key  personnel  at  the 
end  of  the  first  two  weeks,  the  end  of  the  first  month,  and 
then  once  a month  until  the  end  of  the  demonstration.  The 
interview  instrument  described  in  Appendix  K will  be  used. 

• Key  personnel  time  logs  will  be  reviewed  each  week  as  they 
are  received  by  PRC  from  the  ESO. 

Based  on  this  information,  recommendations  for  changes  will  be  de- 
veloped as  needed,  implemented  after  approval  by  the  Communications  Of- 
ficer or  Division  Officer,  and  monitored.  At  the  end  of  the  demonstra- 
tion, guidelines  will  be  prepared  for  the  number  and  type  of  personnel 
assigned  to  an  operational  CMI  system  and  their  tasks. 

F.  Personnel  Training  Requirements 

This  objective  is  concerned  with  determining  the  training  needed 
for  site  staff  supporting  CMI  courses.  An  assessment  will  be  made  of 
the  time  and  activities  in  the  demonstration  that  might  appropriately 
be  classified  as  training  for  the  supporting  personnel.  This  will 
include  the  preparation  of  the  LS  and  the  time  spent  by  the  CMI  advisor 
in  dealing  with  support  personnel.  Inadequacies  and  effective  elements 
of  the  training  will  both  be  identified,  and  recommendations  made  for 
training  to  support  an  operational  CMI  system. 

The  demonstration  has  been  set  up  so  that  the  LS  will  receive  BE/E 
Instructor  training,  and  the  LS  and  CMI  advisor  will  train  the  watch 
supervisors  and  CTs  to  carry  out  their  responsibilities.  Time  spent  by 
the  LS  and  CMI  advisor  with  the  watch  supervisors  and  ETs  will  count 
towards  training.  To  gather  this  data  the  LS  and  CMI  advisor  will 
maintain  logs  from  their  time  of  arrival  at  Stockton.  They  will  record 
the  particular  activity  and  time  spent  in  training  the  others,  includ- 
ing all  staff  orientation  sessions  and  walkthroughs.  Copies  of  the 
logs  will  be  submitted  to  PRC  via  the  ESO  at  the  end  of  the  preparation 
stage  and  on  a weekly  basis  during  the  demonstration. 
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The  information  from  the  logs  will  be  supplemented  with  infor- 
mation from  the  observation  of  procedures  and  the  periodic  interviews 
*'  (Appendix  K)  by  the  PRC  staff. 

•X 

At  the  end  of  the  demonstration  the  following  information  and  data 
will  be  obtained: 

• A summary  of  training  time  for  all  personnel 

• Problems  of  inadequate  training  and  areas  of  satisfaction 

i”  will  be  identified 

From  this  an  assessment  will  be  made  of  the  adequacy  of  the  CMI 
• advisor's  role  and  the  need  for  more  formal  key  personnel  training  pro- 

grams. Recommendations  will  be  made  on  the  amount  and  type  of  train- 
ing, and  the  role  of  the  CMI  advisor  in  setting  up  a new  activity. 

G.  Organization  and  Management 

This  objective  is  concerned  with  determining  the  organization  and 
management  structure  needed  for  successful  operation  of  onsite  CMI 
courses.  There  are  three  considerations: 

• The  existing  structure  at  a base  or  on  board  ship 

• The  structure  needed  to  support  CMI 

• The  degree  to  which  the  two  can  accommodate  each  other 

In  this  demonstration  the  command  structure  for  the  Fleet  Center 
at  Stockton  is  shown  (Figure  III-6),  with  the  position  of  the  LS  indi- 
cated. As  the  demonstration  progresses,  changes  will  be  made  in  these 
organization  and  management  structures  to  eliminate  or  at  least  allevi- 
ate problems.  It  will  be  necessary  to  monitor  M ose  changes.  By  un- 
derstanding why  the  changes  were  made  and  how  well  the  final  system 
functions,  inferences  can  be  drawn  on  the  organization  and  management 
structure  needed  for  operational  use  of  CMI. 
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Figure  m 6.  NAVCOMMSTA  Stockton  Organization 
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The  personnel  time  data  and  activity  descriptions  entered  in  the 


key  personnel  logs  will  show  what  is  required  to  operate  CMI , and  how 
it  can  interfere  with  routine  operations.  Additional  insight  into  why 
different  elements  of  the  CMI  organization  and  management  system  are 
desirable  or  not  desirable  will  be  obtained  from  PRC  observations  and 
interviews  of  key  personnel  and  students  using  the  package  from  Appen- 
dix K.  Specifically,  individual  opinions  will  be  sought  with  regard  to: 

• Lines  of  authority  and  the  level  at  which  decisions  are  made 
on  CMI  procedures 

• Degree  of  participation  in  decisionmaking  by  each  of  the  key 
personnel 

• Flow  of  information  up  through  the  chain  of  command  to  keep 
decisionmakers  apprised  of  the  status  of  the  demonstration 
and  of  problems  needing  resolution 

• Flow  of  information  to  personnel  responsible  for  carrying  out 
the  decisions 

• Speed  at  which  problems  are  identified,  decisions  made,  and 
solutions  implemented 


The  information  sought  will  vary  with  the  level  of  authority.  The 
following  people  will  be  asked  to  submit  to  interviews  on  these  issues: 

• Commanding  Officer,  NAVCOMMSTA  Stockton 

• Communications  Officer 

• Division  Officer 

• Radioman  watch  supervisors.  Leading  Chiefs,  Chiefs  of  the 

watch 
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• CMI  advisor 

• Students 

The  base  commanding  officer,  the  Communications  Officer,  the  Divi- 
sion Officers,  leading  chiefs,  chief  of  the  watch,  and  watch  super- 
visors will  be  interviewed  to  determine  their  level  of  satisfaction, 
and  specific  problems  and  suggested  solutions  concerning: 

• The  reporting  to  them  of  the  status  of  the  demonstration 
status  and  any  problems 

• The  workload  imposed  on  their  staff  by  the  CMI  program 

• The  degradation,  if  any,  of  normal  station  operations 

• The  speed  at  which  decisions  to  change  the  CMI  operation  are 
implemented 

• Any  other  problems  they  encounter 

The  ESO,  DTC/POs,  LS,  and  CMI  advisor  will  be  interviewed 
concerning: 

• Their  ability  to  raise  problems  or  opportunities  for  change 
and  receive  approval 

• The  priority  given  CMI  matters  in  general 

• The  amount  of  time  given  to  prepare  for  changes  in  the  CMI 

operation  or  changes  in  routine  operations  affecting  their 
work  in  the  demonstration 

• Their  participation  in  routine  decisions  regarding  CMI 

These  people  will  also  be  asked  to  identify  other  problems,  and  suggest 
solutions. 

The  ETTs  will  be  asked  the  same  questions  as  the  ESO,  DTC/POs,  the 
LS,  and  the  CMI  advisor.  The  questions,  however,  will  address  the  ETs 


I 


73 


TAEG  Report  No.  49 


,,  duties  in  maintenance  as  well  as  in  substituting  for  the  LS  when  the 

■.  latter  is  off  duty.  The  ETs  will  also  be  asked  to  assess  the  impor- 

tance  of  their  losing  time  from  normal  duties. 

Students  will  be  asked  the  same  questions  as  the  ETs,  with  empha- 
sis on  the  changes  to  the  student's  watch  duties.  They  will  be  asked 
to  assess  the  importance  of  the  reduction  in  normal  duties,  and  to  sug- 
gest changes  related  to  scheduling  or  any  other  aspect  of  the  CMI  or- 
ganization and  management  structure. 

V 

At  the  end  of  the  demonstration  an  assessment  will  be  made  of  the 
effectiveness  of  the  organization  and  management  structures  used  during 
the  demonstration.  Requirements  for  a structure  to  support  an  opera- 
tional CMI  system  will  be  inferred. 

H . Space  Requirements  and  Operational  Procedures 

This  objective  is  concerned  with  the  extent  to  which  day-to-day 
student  and  CMI  staff  procedures  and  the  facilities  enhance  or  hinder 
the  students'  progress.  In  the  periodic  interviews  to  be  held  by  the 
PRC  staff,  the  ESO,  ETs,  LS,  CMI  advisor,  and  students  will  be  asked  to 
assess  the  adequacy  of: 

! 

• Learning  center  space  and  facilities,  especially  the  use  of 

the  conference  room  tables  for  study,  lab  work,  and  exams  j 

• The  scheduling  of  study,  lab  work,  and  exeuns,  including  use  | 

of  materials  I 

! 

• The  scheduling  of  interaction  with  the  LS  and  ETs 

• The  tasks  and  their  scheduling  for  ETs 

1 

• The  procedures  for  keeping  student  and  key  personnel  logs. 
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Figure  III-7  shows  the  layout  of  the  conference  room  at  the  Stockton 
Complex  with  the  location  of  storage  cabinets  for  CMI  material,  the 
LS's  desk,  and  the  Opscan/  Terminet  cluster. 


Figure  111-7.  Stockton  CMI  Demonstration  Learning  Center 


The  interview  responses  will  be  reviewed  by  PRC  staff  and  an  as- 
sessment made  of  which  characteristics  are  important  for  successful 
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operation  of  CMI  onsite.  Inferences  will  be  drawn  for  large  scale 
operation  of  CMI. 

I . Equipment,  Maintenance,  Spare  Parts,  and  Logistics  Requirements 
This  objective  is  concerned  with  matters  of  smooth  and  effective 
operation,  and  the  requirements  in  terms  of  the  type  and  number  of 
pieces  of  equipment,  maintenance  response  time,  availability  of  spare 
parts,  and  the  ease  of  replacing  stocks  of  spares  and  expendable  sup- 
plies. The  costs  of  these  are  considered  in  the  economic  analysis. 

Here  the  emphasis  is  on  the  acceptability  of  the  operation  to  the  per- 
sonnel. 

The  maintenance  activities  are  as  follows: 

• Communication  line  and  Opscan/Terminet  cluster  under  contract 

• Lab  equipment  by  ETs  as  part  of  their  watch  duties 

• Microfiche  readers  by  the  local  distributor  at  Stockton 

Expendable  supplies  will  be  ordered  by  the  ESO  as  needed,  following  the 
established  routine  at  Stockton. 

The  observations  and  periodic  interviews  conducted  by  PRC  staff 
address  the  issue  of  personnel  satisfaction  with  the  demonstration  plan 
for  equipment,  maintenance,  spare  parts,  and  logistics.  In  addition, 
each  week  the  PRC  staff  will  collect  the  following  data  from  the  main- 
tenance logs  and  key  personnel  time  logs; 

• Response  time  of  maintenance  technician 

• Total  down  time 

• Number  of  instances  and  the  delay  resulting  from  inadequate 
spares 

• Unused  spares 


76 


i 


TAEG  Report  No.  49 

• Number  of  instances  of  inadequate  supply  of  expendables, 
Opscan  test  sheets,  and  administrative  forms  (Pi,  P2,  and 
P3) ; Terminet  paper;  time  before  new  delivery 

During  the  demonstration  the  PRC  staff  will  document  the  proce- 
dures and  specific  activities  for  equipment,  maintenance,  spares,  and 
logistics.  The  information  and  data  will  include  maintenance  response 
time,  down  time,  use  or  lack  of  use  of  spares,  spares  shortages,  and 
shortages  of  expendables.  An  assessment  of  the  level  of  satisfaction 
or  dissatisfaction  with  the  demonstration  procedures  will  be  obtained 
from  the  PRC  observations  and  interviews.  From  this,  requirements  for 
onsite  delivery  of  CMI  will  be  derived  and  an  assessment  made  of  how 
they  might  change  as  the  scale  of  the  CMI  effort  changes. 

J.  Conclusion 

The  research  design  described  here  will  provide  the  data  and 
analytical  procedures  to  achieve  the  objectives  of  the  demonstration. 
The  scheduling  of  these  research  activities  and  the  assignment  of  re- 
sponsibilities to  PRC  and  Navy  personnel  are  described  in  the  next 
chapter. 
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Chapter  IV 
DEMONSTRATION  DESIGN 


A.  Introduction 

The  demonstration  can  be  considered  as  a three  part  operation; 

• The  CMI  course,  with  the  activities  of  students  and 
supporting  personnel 

• Research  activities,  consisting  of  data  collection  and 
analysis 

• Support  activities,  which  put  into  place  the  physical 
systems,  equipment,  and  material  needed  to  operate  the 
demonstration 

For  some  people  participating  in  the  project  there  is  overlap  be- 
tween these  areas.  Generally,  however,  they  represent  a clear  separ- 
ation of  functions.  Within  each  area,  separate  treatment  is  given  to 
the  activities  required  in  the  preparation,  conduct,  and  evaluation  of 
the  demonstration.  This  chapter  addresses  each  of  these  areas,  provi- 
ding a component  by  component  description  of  how  the  demonstration  will 
be  made  to  happen.  Where  necessary  the  details  of  day-to-day  activi- 
ties are  described  in  terms  of  what  will  be  done,  who  will  do  it,  when, 
and  how.  A comprehensive  description  showing  how  the  pieces  fit  to- 
gether is  given  in  Chapter  VI,  "Demonstration  Master  Plan." 

B.  Operations 

This  section  describes  specific  activities  which  must  be  carried 
out  during  preparation,  demonstration,  and  evaluation  in  order  to  offer 
the  Oil  BE/E  and  correspondence  courses. 
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1 . Preparation 

The  activities  required  during  the  preparation  phase  are: 

• Select  CMI  and  correspondence  students 

• Assign  and  train  an  LS  for  the  demonstration 

• Assign  a CMI  advisor 

• Conduct  an  orientation  session  at  Stockton  for  site 
staff  who  will  support  the  demonstration 

• Schedule  the  activities  of  key  site  personnel  and 
students 

• Provide  a student  orientation  session  and  a walkthrough 
of  the  communication  system  operation  for  the  students 
and  key  site  personnel 

a.  Selection  of  CMI  and  Correspondence  Students 

The  ESO  is  responsible  for  selecting  and  providing  PRC 
with  a list  of  students  who  have  volunteered  to  participate  in  the  BE/E 
Course  File  69  CMI  course  at  Stockton.  Student  volunteers  must  also  be 
selected  for  the  correspondence  version  of  the  BE/E  course. 

The  ESO  is  responsible  for  making  the  program  known  and 
available  to  all  RM  personnel  who  might  be  interested.  Once  students 
have  indicated  that  they  wish  to  participate  in  the  CMI  or  corres- 
pondence course,  it  will  be  the  responsibility  of  the  ESO  to  have  each 
individual  sign  a standard  Naval  permission  form  which  describes  the 
nature  of  the  research,  and  indicates  to  the  volunteer  the  right  to 
disenroll  in  the  project  should  it  become  necessary. 

It  will  also  be  the  responsibility  of  the  ESO  to  collect 
and  tabulate  the  data  required  for  each  CMI  student  for  registration: 
name,  social  sejurity  number,  branch  of  service,  classification,  year 
of  birth,  number  of  years  of  schooling.  Armed  Forces  Qualification  Test 
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(AFQT)  scores,  and  Basic  Test  Battery  fBTB)  scores.  It  will  be  neces- 
sary to  have  the  same  data  tabulated  for  the  correspondence  students. 

On  July  11  the  ESO  will  submit  to  PRC  a preliminary  list 
of  CMI  volunteers  with  the  registration  data,  but  with  the  names  and 
social  security  numbers  covered.  These  data  are  to  be  used  in  a trial 
run  by  MIISA  Memphis  of  the  BE/E  CMI  prediction  program.  An  estimate 
of  completion  times  for  the  Stockton  students  will  be  secured  by  July 
15  and  used  to  refine  the  projected  schedules.  By  August  8,  the  stu- 
dent lists  will  be  finalized  and  the  ESO  will  prepare  the  needed  regis- 
tration information  using  the  student  registration  card  format  shown  in 
Appendix  H.  This  information  is  to  be  mailed  to  MIISA  San  Diego  with  a 
copy  provided  to  PRC  staff.  The  latter  will  be  at  Stockton  from  August 
8 through  the  second  week  in  September.  The  final  estimate  of  comple- 
tion times  will  be  made  August  12,  when  the  students  are  registered. 

Beginning  July  18  the  ESO  will  survey  outlying  site  per- 
sonnel to  determine  who  will  take  the  correspondence  course.  The  list 
will  be  finalized  by  August  8. 

b.  Assignment  and  Training  of  LS 

It  will  be  the  responsibility  of  CNTECHTRA  to  assign  and 
train  an  LS  for  the  demonstration.  The  LS  selection  and  training  will 
be  ordered  by  the  Electronics  TPC,  with  selection  by  July  11  and  train- 
ing to  be  initiated  July  18.  The  LS  should  have  an  ET  rating,  and 
should  be  able  to  spend  seven  months  in  Stockton.  If  it  becomes  impos- 
sible to  select  an  LS  who  can  stay  on  location  for  the  entire  demon- 
stration, it  will  be  necessary  for  CNTECHTRA  to  select  more  than  one  LS 
in  order  to  maintain  a continuous  full-time  position  at  the  Stockton 
site. 
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c.  Selecting  and  Assigning  a CMI  Advisor 

CNTECHTRA  is  responsible  for  selecting  and  assigning  a 
CMI  advisor  by  July  15  to  arrive  at  Stockton  no  later  than  August  20. 
The  advisor  should  be  someone  who  has  considerable  experience  in  the 
CMI  training  progreim  and  is  knowledgeable  in  the  electronics  field. 

This  person  will  be  required  to  oversee  the  entire  operational  set-up 
as  well  as  provide  ongoing  apprisal  of  the  operation  during  the  first 
weeks  of  the  demonstration. 

d.  Staff  Orientation  at  Stockton 

On  August  22  PRC  will  hold  an  orientation  meeting  at  a 
designated  room  in  the  administration  building  at  Stockton  with  the  LS, 
CMI  advisor,  ESO,  DTC/POs,  Division  Officers,  Communications  Officer, 
and  the  Base  Commander.  The  purpose  of  this  orientation,  expected  to 
require  one  to  two  hours,  is  to  brief  all  key  demonstration  personnel 
on  the  status  of  the  demonstration  design.  Scheduling  of  key  personnel 
and  students  and  the  activities  to  be  carried  out  by  key  personnel  will 
be  discussed.  The  orientation  session  will  be  chaired  by  the  principal 
investigator  from  PRC. 

Beginning  on  Monday  afternoon,  August  22,  and  continuing 
throughout  the  week,  meetings  of  about  three  hours  will  be  held  on 
watch  time  with  the  senior  maintenance  personnel.  These  are  the  ETs 
who  will  substitute  for  the  LS  while  the  latter  is  off  duty.  The  ETs 
will  also  maintain  the  laboratory  equipment  during  the  demonstration. 
PRC  staff  will  also  be  responsible  for  this  session  with  assistance 
from  the  LS  and  the  CMI  advisor.  Since  the  ETs  will  serve  as  resource 
persons,  it  will  be  important  to  determine  how  well  acquainted  they  are 
with  the  course  materials.  At  a minimum,  two  hours  of  this  orientation 
should  be  devoted  to  a review  of  each  module  topic  area  with  the  LS 
serving  as  the  leader  of  this  phase.  The  remainder  of  the  orientation 
will  be  devoted  to  acquainting  the  ETs  with  their  equipment  maintenance 
responsibilities. 
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e.  Scheduling  of  Activities 

Using  the  initial  estimate  of  time-to-complet ion  for  the 
CMI  course  received  August  15,  the  ESO  and  PRC  staff  will  develop  a 
tentative  schedule  for  students  and  key  personnel  by  August  19,  inclu- 
ding review  mretings.  On  the  morning  of  August  29  an  operational 
scheduling  session  will  be  held  in  the  morning  in  the  conference  room 
at  the  Stockton  blockhouse.  The  LS , CMI  advisor,  PRC  staff.  Division 
Officers,  Communications  Officer,  ESO,  and  DTC/POs  from  Stockton  are 
required  for  this  session.  The  tentative  schedule  prepared  by  the  ESO 
and  PRC  staff  will  be  received.  The  end  product  will  be  schedules  for 
the  LS  and  ETs  over  the  entire  demonstration,  monthly  project  review 
meetings  for  key  site  personnel  and  testing  and  interviews  by  PRC  staff. 

CMI  student  schedules  will  be  set  in  meetings  with  the 
ETTs  over  the  week  of  August  29  to  September  2,  beginning  the  afternoon 
of  the  29th.  The  PRC  staff,  the  LS , the  CMI  advisor,  and  the  Division 
Officers  will  participate. 

On  the  morning  of  August  30  an  operational  scheduling 
session  will  be  held  for  the  correspondence  course  in  the  conference 
room.  The  ESO,  the  Communications  Officer,  Officers-in-Charge,  and 
DTC/POs  of  the  outlying  sites  of  the  Stockton  activity,  and  the  PRC 
staff  will  attend  this  morning  session  to  schedule  the  course  work  for 
students  who  will  be  taking  the  BE/E  corresp>ondence  course  at  the  com- 
ponent sites.  The  session  will  also  produce  an  estimate  of  the  in- 
volvement of  the  ESO  and  DTC/POs  who  will  be  handling  student  tests, 
monitoring  students,  and  advising  as  needed.  The  course  materials  will 
be  presented  to  the  DTC/POs  by  the  ESO  for  dissemination  to  the 
students . 
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f . Student  Orientation,  and  Student  and  Key  Personnel 
Walkthrough 

Beginning  the  afternoon  of  August  30  and  continuing 
through  September  2,  the  students,  ETs,  DTC/POs,  and  the  ESO  will  be 
introduced  to  the  course  and  briefed  on  the  operation  of  the  Opscan/ 
Terminet  cluster  and  the  use  of  materials  from  the  learning  center. 

The  schedule  of  the  walkthrough  will  have  been  determined  in  the  work- 
shop of  August  29  and  coordinated  so  as  not  to  conflict  with  the  opera- 
tional schedule  meeting.  The  US  and  Oil  advisor  will  be  required  to 
conduct  these  sessions  with  assistance  from  the  PRC  staff.  A brief 
meeting  of  about  20  minutes  will  be  held  with  the  students  to  describe 
the  course.  Then,  there  will  be  a pilot  run  of  the  communication  sys- 
tem, with  selected  student  volunteers  taking  a sample  test,  feeding  the 
test  answer  sheet  into  the  Opscan,  and  receiving  feedback  on  the 
Terminet  printer.  Time  will  be  spent  to  answer  student  questions. 

2.  The  Demonstration 

This  section  presents  operational  requirements  and  procedures 
to  be  followed  during  the  conduct  of  the  demonstration  for  the 
following: 

• Daily  and  weekly  routine  for  the  CMI  course 

• CMI  student 

• Learning  Supervisor 

• ETs 

• ESO 

• DTC/POs 

• Communications  Officer 

• Division  Officers 

• The  CMI  advisor 

• Daily  and  weekly  routine  for  the  correspondence  course 
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• Correspondence  student 

• DTC/POs 


a.  Stockton  CMI  Course 

Daily,  Weekly,  and  Monthly  Routine  for  the  CMI 

Course 

This  section  provides  an  overall  view  of  demonstra- 
tion procedures.  The  routine  operation  involves  the  students,  the  LS 
and  the  ETs.  Throughout  the  demonstration  the  students  will  spend  an 
average  of  10  hours  per  watch  string.  Since  Stockton  is  on  a rotating 
watch  bill  as  shown  in  Figure  IV-1,  course  activity  will  be  taking 


MARCH 

DATE: 

DAY 

01 

T 

02  03  04  05  06  07  08  09  10  1 1 

WTFSSMTWTF 

12 

S 

13  14 

S M 

15  16  17 

T W T 

18  19  20  21 

F S 5 M 

22  23  24  25  26  27  28  29  30  31 

TWTFSSMTWT 

Ml  0 

1 

2 2 

3 

3 

4 

4 112 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 

4 1 

1 

2 

2 

3 3 

4 4 

DAY: 

2 

3 3 

4 

4 

1 

12  2 3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 

4 

1 

1 2 

2 

3 

3 

4 4 

1 1 

EVE 

1 

1 2 

2 

3 

3 

4 4 11 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 4 

1 

1 

2 

2 3 

3 4 

APRIL 

OATE 

01 

02  03  04  05  06  07  08  09  1 0 

11 

12 

13 

14 

15  16  17 

18  19  20  21 

22  23  24  25  26  27  28  29  30 

DAY 

F 

S S 

M 

T 

w 

T F S S 

M 

T 

w 

T 

F 

s 

S 

M 

T 

w 

T 

F 

s s 

M 

T 

w 

T F 

S 

MID 

1 

1 2 

2 

3 

3 

4 4 11 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 4 

1 

1 

2 

2 3 

3 

DAY. 

2 

2 3 

3 

4 

4 

112  2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 

4 

1 1 

2 

2 

3 

3 4 

4 I 

EVE 

4 

1 1 

2 

2 

3 

3 4 4 1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 4 

4 

1 

1 

2 2 

3 ' 

i 

MAY 

DATE: 

01 

02  03  04  05  06  07  08  09  1 0 1 1 

12 

13 

14 

15 

16 

17 

18 

19  20  21 

22  23  24  25  26  27  28  29  30  31  I 

DAY 

s 

M T 

W 

T 

F 

S S M T 

W 

T 

F 

s 

S 

M 

T 

w 

T 

F 

S 

S 

M T 

W 

T 

F 

S S 

M T 

MID 

4 

4 1 

1 

2 

2 

3 3 4 4 

1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 3 

4 

4 

1 

1 2 

2 3 

DAY 

1 

1 2 

2 

3 

3 

4 4 11 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 

4 4 

1 

1 

2 

2 3 

3 4 

EVE 

3 

4 4 

1 

1 

2 

2 3 3 4 

4 

1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 3 

3 

4 

4 

1 1 

2 2 

JUNE 

DATE 

01 

02  03  04  05  06  07  08  09  10  1 1 

12 

13  14  15 

16  17 

18  19  20  21 

22  23  24  25  26  27  28  29  30 

DAY 

W 

T F 

S 

S 

M 

T W T F 

s 

S 

M 

T 

W 

T 

F 

S 

s 

M 

T 

w 

T F 

S 

s 

M 

T W 

T 

MID 

3 

4 4 

1 

1 

2 

2 3 3 4 

4 

1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 3 

3 

4 

4 

1 1 

2 

DAY 

4 

1 1 

2 

2 

3 

3 4 4 1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 

2 

3 

3 4 

4 

1 

1 

2 2 

3 

EVE 

3 

3 4 

4 

1 

1 

2 2 3 3 

4 

4 

1 

1 

2 

2 

3 

3 

4 

4 

1 

1 

2 2 

3 

3 

4 

4 1 

1 

Figure  IV- 1.  Sample  Watch  Bill 
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place  over  a 24-hour  day,  seven  days  a week.  This  activity  has  to  be 
modified,  however,  from  what  occurs  at  a schoolhouse: 

• The  CMI  computer  is  not  available  around  the  clock.  The  041 
concentrator  in  San  Diego  is  the  controlling  factor  in  allow- 

■ ing  access  to  the  CMI  computer  in  Memphis.  The  concentrator 
is  on-line  from  0600  to  1830  Pacific  time,  Monday  through 
Friday,  so  it  is  only  between  those  hours  and  on  those  days 
that  normal  student-computer  interaction  procedures  can  be 
followed.  During  this  time  students  arriving  at  the  learning 
center  will  submit  a Pi  administrative  form  through  the 
Opscan  to  log  in.  Any  tests  taken  by  the  student  during  that 
time  will  be  entered  in  the  Opscan  as  soon  as  they  are  com- 
pleted by  the  student.  The  computer  in  Memphis  will  score 
the  tests  and  direct  the  student  to  the  next  assignment  by  a 
message  printed  on  the  Terminet.  At  the  end  of  the  study 
period  the  student  is  to  submit  a P2  administrative  form  to 
log  out.  CMI  experience  shows  that  students  often  forget  to 
log  out  with  a P2  form.  Therefore,  it  is  important  that  the 
students  maintain  time  cards.  The  LS  can  refer  to  the  cards 
and  enter  a P3  form  to  correct  the  student's  file  if  a P2  was 
not  used  to  log  out.  When  there  is  no  connection  to  the  CMI 
computer  in  Memphis,  the  students  can  work  up  to  the  point  of 
completing  an  exam. 

• Only  one  LS  will  be  available,  and  only  Monday  through 
Friday,  0800  to  1600. 

• At  any  time  of  day  there  are  ETs  on  duty.  They  will  be  on 
call  to  handle  administrative  and  course  content  questions  in 
the  absence  of  the  LS.  In  addition,  the  ETs  are  responsible 
for  maintenance  of  lab  equipment. 

• No  more  than  half  the  students  at  a time  will  be  permitted  to 
leave  their  duty  station  at  a time  during  their  watch  to  work 
on  the  course.  This  is  a requirement  to  avoid  impairing  com- 
munication activities. 

• Since  the  evening  watch  is  very  busy,  students  will  not  be 
able  to  work  on  the  CMI  course  when  they  have  that  watch.  To 
make  up  for  the  course  time  lost  from  the  evening  watch, 
students  can  spend  three  hours  on  the  mid  watches  when  the 
work  load  is  low.  During  day  watch  students  can  be  expected 
to  spend  two  hours  on  the  course,  for  a total  of  10  hours  of 
course  work  per  watch  string.  Each  watch  string  consists  of 
six  watches  plus  80  hours  off  covering  eight  complete  days. 
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Figure  IV-2  shows  the  effect  of  these  operational 
factors  on  personnel  responsibilities  for  a typical  weekday.  The  ETs 
participate  in  the  demonstration  only  in  the  absence  of  the  LS,  and  are 
responsible  for  both  administrative  matters  and  technical  questions. 
Their  maintenance  responsibility  extends  across  the  whole  day.  when 
needed  to  monitor  lab  work  or  performance  tests,  they  will  spend  a ma- 
jor portion  of  their  watch  in  the  learning  center. 

When  the  LS  arrives  in  the  morning,  he  will  enter 
three  sheets  through  the  Opscan  for  each  student:  The  first  is  a Pi 
form  stating  an  artificial  student  start  time  such  that  the  period  be- 
tween that  time  and  the  time  the  LS  enters  the  sheets  into  the  Opscan 
equals  the  actual  study  time  spent  by  the  student.  The  second  form  is 
the  test  sheet,  and  the  third  is  a P2  form  for  logging  the  student  out. 


On  a weekly  basis,  routine  activities  will  vary  as 

follows: 

• Saturday  and  Sunday — The  LS  will  not  be  on  duty,  so  the  re- 
sponsibilities of  the  ETs  will  be  as  shown  in  Figure  IV-2  for 
the  mid  and  day  watches.  The  students  will  follow  the  appro- 
priate time  sheet  entry  and  test  procedures  when  the  concen- 
trator is  down  (1830-0600  Pacific  time) . 

• Monday — The  LS  should  receive  the  learning  center  roster  from 
MIISA  San  Diego  which  summarizes  student  progress.  Consul- 
tations would  have  to  be  arranged  for  students  who  are  not 
doing  well. 

• Friday — LS  and  ET  cost  and  time  logs  will  be  collected  by  the 
LS  and  sent  to  PRC.  MIISA  San  Diego  will  run  and  mail  the 
learning  center  roster.  During  the  first  month  rosters  will 
be  mailed  on  Tuesday  as  well. 


■ All  study,  test  and  laboratory  materials,  student 
time  sheets,  and  completed  test  sheets  will  be  kept  in  locked  storage 
cabinets  in  the  learning  center.  The  LS  will  keep  the  cabinets  un- 
locked while  working  in  the  learning  center.  At  all  other  times  the 
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cabinets  will  be  locked  with  keys  available  in  the  master  key  box  in 
Tech  Control.  A log  book  will  be  kept  at  the  box  for  students  or  ETs 
to  sign  out  when  using  a key. 

Each  month,  a review  meeting,  chaired  by  the  LS, 
will  assess  the  effectiveness  of  demonstration  procedures  and  sched- 
ules, and  revise  them  as  appropriate.  The  participants  will  be  the  LS, 
ESO,  DTC/POs,  ETs,  Division  Officers,  and  Communications  Officer.  Two 
of  these  meetings  will  be  held  in  the  first  two  weeks  to  assure  a good 
start  for  the  demonstration.  These  first  two  meetings  will  differ  from 
the  general  format  in  that  the  CMI  advisor  will  participate,  and  the 
PRC  staff  will  develop  the  agenda  and  an  assessment  of  proposed  changes 
in  procedure  or  schedule.  The  monthly  meeting  agenda  will  be  prepared 
by  the  LS. 


(2)  The  CMI  Student 

The  general  study  procedure  for  the  CMI  student  be- 
gins by  reporting  to  the  learning  center  at  the  scheduled  time  during 
the  watch.  After  checking  in  with  the  LS  or  designated  ET,  the  student 
begins  study  with  the  assigned  materials. 

When  the  student  arrives  at  a point  of  being  ready 
for  a module  test,  the  microfiche  test  cards  will  be  provided  by  the  LS 
or  ET.  After  successfully  completing  a module  test,  the  student  will 
receive  instructions  on  the  Terminet  to  proceed  to  the  next  module. 

The  LS  or  ET  will  provide  assignment  materials  for  the  next  module  and 
will  keep  a record  of  student  progress  in  each  student's  file  using  the 
form  from  Appendix  L.  The  student  will  keep  a study  procedures  file 
(shown  in  Appendix  L)  as  a backup,  and  will  be  expected  to  maintain 
reasonable  progress  on  course  materials  based  on  CMI  computer  projec- 
tions. Should  any  student  fall  30  percent  behind,  he  or  she  will  be 
counseled  by  the  LS. 
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Assuming  these  procedures,  a time  estimate  for  stu- 
dent progress  through  the  CMl  course  is  presented  in  Figure  IV-3.  The 
scenario  is  based  on  estimates  derived  from  the  Course  File  69  BE/E 
schoolhouse  data  for  the  slowest  student.  The  reader  should  remember 
that  no  data  currently  exists  for  estimating  projected  completion  times 
for  the  BE/E  course  in  an  operational  training  environment.  This  fig- 
ure is  based  on  the  schoolhouse  data  for  students  requiring  an  average 
of  200  hours  to  complete  the  course.  The  figure  also  assumes  some 
"start  up"  time  for  each  module  or  lab  performance  activity  and  ac- 
counts for  some  vacation  time  during  official  holiday  periods.  Spe- 
cifically, the  following  assumptions  were  made  in  preparing  the 
chart: 

• The  dates  are  estimated  for  the  last  group  to  start  the 
course,  the  group  whose  watch  string  does  not  begin  until 
September  9. 

• Three  days  are  taken  off  at  Thanksgiving. 

• Nine  days  are  taken  off  at  Christmas. 

• After  returning  from  leave,  students  rejoin  their  watch  sec- 
tion the  next  time  it  is  on  duty. 

Based  on  these  assumptions,  the  last  student  is  expected  to  finish  by 
February  24. 


(3)  The  Learning  Supervisor 

The  LS  is  required  to  perform  the  following  activi- 
ties on  weekdays  between  0800  and  1600  hours: 

• Advise  students  on  the  use  of  the  Opscan  and  Terminet,  the 
use  of  test  papers,  maintenance  of  the  time  log  and  schedul- 
ing of  their  time,  and  administer  examinations. 

• When  requested,  assist  students  on  substantive  questions  re- 
lating to  course  content,  their  progress,  and  remedial  *»ork 
where  needed.  This  procedure  is  particularly  crucial  during 
those  times  indicated  in  Figure  IV-3  as  difficult. 


Figure  I V-3.  Time  Estimate  For  Completion  Of  CM!  Course 
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Maintain  student  perfornance  files  and  schedule  students  to 
use  the  learning  center  and  lab  equipment,  administer  tests. 

A file  cabinet  for  performance  files  and  other  administrative 
material  will  be  stationed  in  the  learning  center.  The  LS 
should  check  student  progress  and  other  administrative  mat- 
ters at  the  beginning  of  each  day. 

Check  computer  time  entries  (log-in  and  log-out)  by  day  stu- 
dents and  check  later  time  data  for  evening  and  mid  watch 
students.  As  indicated  earlier,  this  is  particularly  impor- 
tant since  students  may  forget  to  log  out  with  the  P2  form  at 
the  conclusion  of  a study  period. 

Maintain  the  stock  of  materials,  including  Opscan  test 
sheets,  administrative  Opscan  sheets  (Pi,  P2,  P3) , expendable 
supplies,  Terminet  paper,  module  tests  and  exams.  There  will 
be  space  available  in  one  of  the  storage  cabinets  for  these 
materials.  The  LS  will  inventory  these  materials  each  Friday. 

Request  and  check  on  maintenance  of  the  Opscan,  Terminet,  and 
lab  equipment  as  needed 

Coordinate  student  and  ET  activities  with  the  ESO,  DTC/POs, 
Communications  Officer,  and  Division  Officers.  A meeting 
JMfll  be  held  once  a month  to  appraise  the  status  of  the  CMI 
training  activities  and  revise  procedures  as  needed.  One 
meeting  will  also  be  held  each  of  the  first  two  weeks  of  the 
demonstration. 


(4)  Senior  Maintenance  ETs 

The  major  functions  of  the  ETs  will  be  to; 

• Substitute  for  the  LS  on  mid  and  day  watches  when  the  LS  is 
off  duty 

• Maintain  lab  equipment  on  an  "on-call"  basis 


The  administrative  functions  include  answering  student  questions  on  leg 
entries,  providing  access  to  course  materials,  administering  exams, 
checking  student  log-in  and  log-out  on  the  Opscan  when  the  computer  is 
on  line,  and  collecting  exam  papers  and  student  study  times  for  entry 
by  the  LS  when  the  computer  is  on-line.  The  content-related  functions 
include  answering  student  questions  on  the  content  of  the  course,  and 
supervising  lab  work  and  performance  tests. 
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FT 


All  exeun  papers  and  student  study  times  that  are 
collected  during  computer  down  times  will  be  placed  in  a file  marked 
for  that  purpose  in  the  learning  center  storage  cabinets.  This  is  to 
be  locked  by  the  ETs  along  with  all  other  storage  cabinets  during  their 
watches  and  the  keys  kept  in  the  designated  box  in  Tech  Control. 

The  ETs  will  maintain  time  records  of  their  activi- 
ties and  will  participate  in  the  monthly  review  meetings. 

(5)  The  Education  Services  Officer 

The  ESO  will  perform  the  following  activities  in 
support  of  the  QMI  course: 

• Establish  student  files  containing  student  registration  data 
for  use  by  LS.  The  LS  will  add  time  cards  and  other  progress 
information  as  appropriate 

• Coordinate  routine  training  activities  with  the  CMI  course 

• Participate  in  the  monthly  review  meetings 

• Assist  the  LS  in  coordinating  with  and  monitoring  students 

• Maintain  time  records  of  his  activities 

(6)  DTC/POs 

The  DTC/POs'  responsibilities  include: 

• Coordinate  Division  training  with  the  CMI  course 

• Participate  in  the  monthly  review  meetings 

• Assist  the  LS  in  working  with  Division  students 

• Maintain  time  records  of  their  own  activities 

(7)  Communications  Officer 

During  the  demonstration  the  Conununi cat ions  Officer 
will  do  the  following: 
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• Coordinate  the  use  of  the  learning  center  with  the  LS.  The 
room  is  to  be  preempted  whenever  needed  for  Fleet  Center 
use.  At  the  August  29  workshop  a schedule  of  routine  Fleet 
Center  meetings  will  be  adopted  so  CMI  activities  can  be 
planned  around  them.  The  Communications  Officer  will  provide 
as  much  advance  notification  to  the  LS  as  possible  for  un- 
expected meetings. 

• Coordinate  with  the  LS,  and  approve  the  scheduling  of  stu- 
dents and  ETs  for  the  respective  tasks.  This  will  be  done 
initially  in  the  August  29  workshop  and  revised  as  needed  in 
the  monthly  review  meetings  with  the  LS  and  other  key  person- 
nel. 

• Serve  as  a liason  between  CMI  training  personnel,  the  Fleet 
Center  staff  and  the  base  commander 

• Maintain  time  records  of  his  activities 


(8)  Division  Officers 

The  Division  Officer  will  be  responsible  for  co- 
ordinating Division  activities  with  the  CMI  operation.  This  will  be 
initially  done  at  weekly  meetings,  and  later  at  the  monthly  meetings. 
Any  unusual  involvement  will  be  logged  and  reported  to  the  LS. 


(9)  The  CMI  Advisor 

During  the  first  few  weeks  of  the  demonstration  the 
CMI  Advisor  will  assist  the  LS  and  other  project  personnel  as  needed. 
The  advisor  will  follow  a part-time  schedule  of  observing  activities 
seven  days  a week  and  will  assist  the  LS  in  refining  operational  proce- 
dures and  solving  problems  as  they  arise.  At  the  conclusion  of  his 
stay  at  the  demonstration  site,  about  September  19,  the  advisor  will 
spend  a day  debriefing  PRC  staff. 

b.  Stockton — Correspondence  Course  Sites 

Individuals  at  the  Stockton  outlying  sites  will  take  a 
correspondence  version  of  the  BE/E  course.  The  sites  are:  Mare 
Island,  Skaags  Island,  Treasure  Island,  Monterey,  Dixon,  Oakland, 
Moffett  Field,  and  Alameda. 
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Since  the  correspondence  course  is  self-taught  and  self 
graded,  the  follwing  personnel  will  participate  in  the  demonstration: 

• The  correspondent  students 

• DTC/POs 

• The  ESO 


(1)  The  Correspondence  Student 

The  correspondence  students  will  spend  an  average 
of  10  hours  per  watch  string  studying  course  material,  thus  completing 
each  lesson  in  two  weeks  or  less.  At  the  end  of  each  lesson,  the  stu- 
dent will  take  a self-graded  test  and  mail  the  test,  along  with  a time 
record,  to  the  ESO.  Upon  satisfying  lesson  requirements,  the  student 
will  move  on  to  the  next  lesson,  thus  completing  the  course  in  about  18 
weeks.  The  last  student  is  expected  to  finish  by  January  20,  1978. 


(2)  DTC/POs 


Upon  requests  by  the  students  or  the  ESO,  the  DTC/ 
POs  will  work  with  students  having  difficulty. 


(3)  The  ESO 

The  ESO  maintains  correspondence  student  files  of 
completed  tests  and  time  records  sent  to  the  ESO  by  students  at  the 
off-base  sites.  These  files  will  also  contain  the  same  personal  and 
aptitude  data  which  is  used  for  registration  of  the  CMI  students.  The 
complete  file  will  be  used  in  evaluating  the  students'  performance  and 
attitudes,  will  be  available  by  January  27. 


3.  Evaluation 

Operational  requirements  during  the  evaluation  phase  of  the 
COMISAT  project  involve  the  LS  and  ESO. 
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a.  The  Learning  Supervisor 

The  primary  responsibility  of  the  LS  during  the  evalua- 
tion phase  will  be  to  participate  in  the  formal  closing  of  the  CMl 
course.  The  LS  will  receive  the  Graduate  Performance  Summary  and  At- 
trition Performance  Summary  data  from  MUSA  San  Diego  and  review  them 
to  determine  that  everything  is  in  order.  The  LS  will  also  chair  a de- 
monstration debriefing  session  with  PRC  staff  and  key  support  person- 
nel. This  meeting  is  to  be  announced  immediately  following  the  gradu- 
ation of  the  last  demonstration  students. 

b.  The  ESQ 

The  ESO  will  participate  in  debriefing  sessions  to  be 
held  following  the  completion  of  the  CMI  course.  The  ESO  will  assist 
in  administering  the  common  performance  exam  to  both  the  correspondence 
and  CMI  students. 

C.  Research  Activities 

The  activities  to  be  carried  out  in  meeting  the  eight  research  ob- 
jectives of  the  demonstration  are  described  here  for  each  objective. 
There  are  separate  discussions  of  preparation,  demonstration,  and 
evaluation  activities  for  each  objective. 

1 . Learning  Effectiveness 
a.  Preparation 

During  the  preparation  phase,  experimental  and  control 
groups  will  be  assembled  to  evaluate  the  effects  of  CMI  onsite  training 
versus  CMI  training  center  training  on  learning  effectiveness.  Back- 
ground information  will  be  gathered  for  evaluation  efforts  and  esti- 
mates of  completion  times  will  be  computed  to  aid  in  scheduling  the 
delivery  of  the  CMI  course. 
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In  the  first  activity  of  the  preparation  phase,  MUSA 
Memphis  will  identify  a pool  of  control  group  subjects  using  Course 
File  69  historical  records  of  course  performance  data.  The  entire  file 
is  to  be  searched  for  records  of  personnel  from  the  E4  to  E6  paygrade 
who  have  taken  the  BE/E  69  course.  By  July  22,  1977,  a hardcopy  of  the 
results  will  be  provided  to  PRC  for  review.  By  July  29,  1977,  a con- 
trol group  of  30  or  more  subjects  will  be  selected.  PRC  will  then 
tabulate  all  of  these  data  on  the  tally  sheet  shown  in  Appendix  H for 
analysis  during  the  evaluation  phase. 

A group  of  experimental  (CMI)  and  correspondence  stu- 
dents will  be  identified  at  the  NAVOOMMSTA  Stockton.  The  ESO  at 
Stockton  will  mail  this  information  on  August  8,  1977  to  MIISA  San 
Diego  for  registration  of  CMI  students  with  copies  to  PRC  on  site  at 
Stockton.  The  following  demographic  data  for  CMI  students  will  be  sup- 
plied by  the  ESO: 

• Social  Security  Number 

• Name 

• Rate 

• Branch  of  Service  {Navy,  Marines) 

• year  of  Birth 

• Years  of  Education 

• Armed  Forces  Qualifications  Test  Score 

• BTB  scores 

MIISA  San  Diego  will  code  the  data  on  the  student  registration  card  for 
the  registration  process  (see  Appendix  H) . After  the  official  regis- 
tration is  completed  by  MIISA  San  Diego,  by  August  12,  a copy  of  the 
learning  center  roster  is  to  be  mailed  from  MIISA  San  Diego  to  the  ESO 
at  Stockton  and  to  PRC  at  McLean,  Virginia. 


97 


TAEG  Report  No.  49 


MIISA  Memphis  will  establish  a learning  center  complex 
I.D.  number  by  August  8,  1977  and  notify  MIISA  San  Diego  by  telephone. 
This  information  is  needed  for  personnel  at  MIISA  San  Diego  to  accom- 
plish the  registration  of  CMI  students  in  the  system.  That  will  be 
done  by  August  12,  1977. 

The  ESO  at  Stockton  will  identify  a group  of  correspon- 
dence students  at  various  NAVCOMMSTA  Stockton  outlying  sites  by  August 
8,  1977.  The  ESO  will  enroll  the  students  in  the  BE/E  correspondence 
course  through  NAVEDTRA  Pensacola.  The  same  background  data  on  level 
of  education  and  ability  used  for  CMI  student  registration  will  be  col- 
lected by  the  ESO  for  the  correspondence  students  and  used  by  PRC  to 
evaluate  correspondence  student  performance. 

b.  The  Demonstration 

During  the  demonstration,  a complete  record  of  all  stu- 
dent time  spent  in  study  and  testing  for  the  CMI  program  must  be  kept. 
For  the  CMI  students,  data  records  are  kept  by  computer  as  described  in 
the  Operations  section,  with  a backup  record  on  a timecard  as  shown  in 
Appendix  J.  This  card  will  contain  a chronology  of  the  student's  study 
program  for  authorized  time  during  his  watch,  any  extra  study  time  he 
spends,  and  lab  and  test  time. 

Correspondence  students  will  record  study  time  and  rela- 
ted activities  during  the  demonstration  in  the  same  manner  as  CMI  stu- 
dents except  the  data  will  not  be  recorded  in  the  computer  system. 

While  this  course  performance  will  not  be  compared  to  CMI  course  data, 
it  will  be  used  when  the  special  examination  scores  are  compared.  The 
ESO  will  maintain  a summary  progress  report  on  those  students  using  the 
form  in  Appendix  J; 
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During  the  first  three  weeks  of  October  1977,  the  Stan- 
ford Diagnostic  Reading  Test  (Form  B,  the  Blue  Level)  will  be  adminis- 
tered by  a PRC  representative  with  the  assistance  of  training  personnel 
at  the  outlying  Stockton  sites  and  at  the  main  base.  The  scoring  and 
coding  of  these  data  for  computer  analysis  will  be  the  responsibility 
of  PRC  staff.  The  measure  of  interest  in  this  test  will  be  the  student 
reading  ability  score  expressed  in  grade  unit  levels. 

The  testing  takes  approximately  two  hours  and  will  be 
given  to  students  on  their  normal  watch.  The  first  testing  at  Stockton 
will  take  place  on  Monday,  October  3,  1977,  with  the  rest  of  the  week 
used  to  test  those  who  were  not  scheduled  for  duty  on  Monday,  or  who 
missed  the  first  session  for  some  other  reason.  For  those  who  are  day 
workers,  testing  will  follow  the  same  procedure  except  they  will  report 
at  some  time  during  their  shifts. 

Because  of  distances  between  outlying  sites  and  the  de- 
sire to  test  students  on-watch,  PRC  staff  will  not  administer  the  test 
directly  at  those  sites.  Instead,  the  PRC  staff  will  meet  with  the 
DTC/POs  and  arrange  for  them  to  test  their  students.  The  PRC  staff 
will  visit  two  sites  per  day  as  follows; 

• Monday  October  10 

0800 — Mare  Island 

1300 — Skaags  Island 

• Tuesday,  October  11 

0800— Moffett  Field 

1300 — Monterey 

• Wednesday,  October  12 

0800 — Oakland 

1300 — Alameda 

• Thursday,  October  13 

0800 — Dixon 
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The  DTC/POs  will  conduct  the  tests  during  the  week  following  the  PRC 
visit.  All  testing  will  be  completed  by  Wednesday,  October  19. 

PRC  will  develop  the  special  examination  for  comparing 
student  achievement  through  the  correspondence  course  with  that  for  CMI 
by  December  15.  The  test  will  be  given  on-watch  by  the  DTC/POs  at 
Stockton  and  the  outlying  sites.  Each  student  will  be  scheduled  to 
take  the  exam  within  two  to  seven  days  after  completing  their  course. 
This  will  allow  some  grouping  to  save  ESO  and  DTC/POs  time.  The  last 
exam  will  be  taken  by  March  3 and  the  results  sent  to  PRC  so  as  to  ar- 
rive by  March  10. 

At  the  conclusion  of  the  demonstration,  when  ell  stu- 
dents in  the  experimental  group  at  Stockton  have  completed  BE/E  course 
requirements,  an  administrative  report  summarizing  each  student's  re- 
corded time  data  is  to  be  sent  by  MIISA  Memphis  personnel  to  the  ESO 
and  the  LS,  and  to  PRC.  The  total  amount  of  time  required  for  each 
student  to  complete  the  course  and  BTB  scores  will  be  coded  from  the 
report  to  the  data  tally  sheet  by  PRC  for  statistical  analysis. 

c.  Evaluation 

Actual  course  completion  time  data  will  be  automatically 
stored  in  the  CMI  computer.  The  ESO  will  assist  PRC  staff  in  tabula- 
ting this  data  on  the  appropriate  ESO  tally  sheet.  Copies  of  the  data 
tally  sheet  are  to  be  filed  with  the  ESO  at  NAVCOMMSTA  Stockton  and  at 
PRC  in  McLean,  Virginia. 

PRC  staff  will  code  course  performance  data  from  the 
data  tally  sheets  onto  computer  cards  for  the  analysis,  and  complete 
the  covariance  analysis  described  In  Chapter  V by  March  24. 

By  March  24,  PRC  staff  will  also  have  completed  the 
analysis  of  the  special  achievement  examination. 
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2.  Attitudes 

a.  Preparation 

The  preparation  phase  begins  with  the  aJTECHTRA  Memphis 
BE/E  TPC  identifying  a group  of  55  students  from  the  training  center  at 
Great  Lakes  and  55  students  from  the  training  center  at  Memphis  to  par- 
ticipate in  a pilot  validation  of  the  attitude  instrument.  The  ar- 
rangements for  this  survey  must  be  made  so  the  validation  can  be 
conducted  the  week  of  July  18,  1977  at  Great  Lakes  and  during  the  week 
of  July  25  at  the  training  center  in  Memphis.  PRC  will  administer  the 
validation  as  well  as  score  and  code  the  data  for  factor  analysis. 

The  procedures  for  collecting  this  data  will  be  the  same 
at  the  two  locations.  The  selected  students  will  report  for  the  survey 
within  one  day  of  completing  the  CMI  course  to  select  the  session  they 
wish  to  attend.  The  survey  will  be  administered  twice  a day,  once  in 
the  morning  and  once  in  the  afternoon,  for  a weekly  total  of  10  ses- 
sions . 


Each  session  will  last  about  an  hour.  During  the  first 
20  minutes  students  will  listen  to  tape  recorded  instructions  and  PRC 
staff  will  answer  questions.  The  survey  itself  will  take  about  40 
minutes.  At  the  end  of  the  hour,  one  student  from  each  session  will  be 
selected  to  stay  for  a two-hour  detailed  oral  interview.  There  will  be 
10  interviews  from  each  of  the  two  locations. 

Scoring  and  coding  of  the  semantic  differential  data  is 
the  responsibility  of  PRC,  with  the  factor  analysis  scheduled  for  the 
week  of  August  1,  1977.  The  validation  will  be  completed  by  August  12. 

The  hour-long  attitude  survey  pretest  will  be  adminis- 
tered by  a PRC  representative  beginning  August  22,  1977.  Scoring  and 


J 

A 


TAEG  Report  No.  49 


coding  of  this  data  will  be  PRC's  responsibility.  No  Inmediate  statis- 
tical analysis  is  required  for  this  pretest  data.  The  data  will  be  re- 
corded on  the  data  tally  sheets  for  later  evaluation. 

The  procedure  for  accomplishing  this  data  collection 
begins  Monday,  August  22,  1977.  During  that  week  CMI  students  at 
NAVCOMMSTA  Stockton  will  report  to  the  training  division  classroom  in 
the  administration  building  at  a specified  time  during  their  regularly 
scheduled  watch.  Day  workers  will  report  to  the  testing  site  at  a 
specified  time  during  their  shift. 

Although  the  attitude  control  group  has  not  yet  been 
determined,  the  procedure  will  be  essentially  the  same,  with  the  pre- 
test administered  during  the  student's  CMI  orientation.  This  testing 
will  be  supervised  by  PRC  staff,  and  will  be  completed  sometime  in 
November. 


b.  The  Demonstration 

During  the  demonstration  phase,  attitude  posttest  data 
will  be  collected  from  CMI  students  as  they  complete  training. 

Posttest  attitude  surveys  at  NAVCOMMSTA  Stockton  will  be  administered 
by  PRC  staff.  This  procedure  requires  about  one  hour  to  complete,  and 
will  be  similar  to  the  pretest. 

Upon  graduation.  Oil  students  will  report  to  the  train- 
ing division  classroom  in  the  administration  building  where  PRC  staff 
will  be  present  to  administer  the  survey  and  interviews.  The  same 
staff  will  score  and  code  the  attitude  posttests  as  they  are  comple- 
ted. The  posttests  will  be  ccmipleted  by  February  28. 

Posttest  attitude  surveys  for  the  control  group  will  be 
administered  at  the  conclusion  of  the  course  in  the  same  room  as  the 
pretest.  The  procedure  will  be  the  same  as  for  the  experimental  group, 
and  are  expected  to  be  completed  by  December  31,  1977. 
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c.  Evaluation 

During  the  evaluation  phase  all  attitude  data  will  be 
prepared  by  PRC  for  analysis  by  the  Biomed  Computer  program.  Since 
there  are  seven  different  attitude  concepts  being  measured  in  the  de- 
monstration, each  will  require  a separate  analysis.  The  analysis  is 
scheduled  to  proceed  immediately  following  the  conclusion  of  the  inter- 
views and  will  be  completed  by  April  14,  1978. 


3.  Economic  Evaluation 

a.  Preparation  for  the  Demonstration 

In  the  preparation  phase,  there  are  two  activities: 

• PRC  will  produce  four  forms  by  August  20: 

— Maintenance  log 

Student  timecard 

— Correspondence  course  student  progress  form 
— Key  personnel  time  log 

(Samples  of  these  four  forms  are  given  in  Appendix  J.) 

• PRC  will  distribute  and  explain  the  use  of  the  CMI  forms  to 
key  Stockton  personnel  at  a workshop  planned  for  August  29. 

• The  LS  and  ETs  will  be  given  the  maintenance  log  and  see  to 
it  that  contractor  maintenance  personnel  fill  out  the  appro- 
priate log  on  call,  and  that  the  ETs  keep  them  up  to  date 
when  they  perform  maintenance. 

• PRC  will  provide  the  ESO  with  the  correspondence  course  stu- 
dent progress  form  on  August  30. 

t 

• PRC  will  distribute  and  explain  the  use  of  the  key  personnel 
time  log  to  the  MIISA  San  Diego  Computer  Center  Director  and 
the  Deputy  Director  of  MIISA  at  Memphis. 

b.  Demonstration  Activities 

The  data  required  for  the  systems  evaluation  will  be  re- 
corded by  key  personnel  during  the  demonstration.  As  described  in 
Chapter  III,  data  collection  mechanisms  currently  at  the  site  will  be 
utilized  and  an  effort  made  to  keep  the  data  collection  requirements  to 
a minimum.  The  data  collection  activities  are  as  follows: 
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• On  the  first  working  day  of  each  month  the  LS  will  send  a 
copy  of  the  maintenance  log  to  PRC  in  McLean,  Virginia. 

• At  the  end  of  the  demonstration,  MIISA  Memphis  will  provide 
PRC  with  a complete  cost  breakdown  of  the  communication  sys- 
tem. The  data  will  be  provided  by  March  10. 

• Student  course  time  will  be  provided  to  PRC  by  MIISA  Memphis 
for  the  CMI  students  by  March  10.  Total  course  time  for  the 
correspondence  students  will  be  sent  to  PRC  by  the  ESO  by  the 
same  date. 

• The  weekly  submission  of  logs  by  key  personnel  will  provide 
the  time  data  needed  for  the  economic  analysis. 

• The  BE/E  TPC  at  ONTECHTRA  will  provide  PRC  at  McLean  with  an 
itemized  costing  of  each  shipment  of  lab  equipment,  micro- 
fiche readers,  microfiche  tests,  texts,  answer  sheets,  ad- 
ministrative forms,  and  Terminet  paper.  The  cost  data  for 
the  August  20  delivery  will  be  sent  to  PRC  at  McLean  by 
August  29.  Later  shipments,  if  needed,  will  be  reported  to 
PRC  within  a week  of  the  delivery.  In  addition,  the  LS  or  ET 
will  note  in  their  logs  any  instances  where  equipment  or  ma- 
terials are  not  available  to  a student. 

• Correspondence  materials  costs  will  t>e  obtained  by  PRC  staff 
from  the  Naval  Education  and  Training  Program  Development 
Center,  Pensacola,  Florida.  A trip  will  be  made  the  week  of 
October  17-21  for  one  or  two  days  of  data  collection. 

c.  Evaluation 

The  results  of  the  demonstration  will  be  used  along  with 
other  data  on  training  costs  to  determine  whether  revision  to  the 
cost-effectiveness  analysis  of  CMl  is  necessary.  A major  extension  of 
the  analysis  will  be  to  assess  the  potential  for  CMI  use  in  the  Defense 
Department  as  a whole. 


By  March  10,  1978,  the  economic  analysis  data  will  be 
available  to  PRC  from  the  demonstration.  From  that  point  until  June  1, 
the  demonstration  results  will  be  assessed  and  conclusions  reached  con- 
cerning the  potential  role  of  CMI  in  Navy  training,  and  in  overall 
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Defense  Department  training.  Recommendations  will  be  made  concerning 
future  Navy  and  Defense  Department  use  of  CMI. 

4.  Personnel  Requirements 

This  objective  is  concerned  with  determining  the  tasks  that 
have  to  be  performed  by  onsite  personnel  and  the  level  of  effort  re- 
quired in  order  to  support  an  effective  CMI  program.  The  personnel  of 
interest  are  the  LS,  ETs,  DTC/POs,  ESO,  Division  Officers,  and  the  Com- 
munications Officer. 

a.  Preparation 

During  the  preparation  for  the  demonstration  the  sched- 
ules and  tasks  of  the  key  site  personnel  will  be  set.  The  workshop  of 
August  29  will  finalize  plans.  This  objective  is  concerned  with  asses- 
sing the  effectiveness  of  those  plans. 

b.  Demonstration 

PRC  staff  will  observe  the  operation  continuously  over 
the  first  two  weeks,  and  then  interview  the  students  and  key  personnel 
to  see  what  might  be  done  to  improve  tasks  or  schedules.  The  interview 
will  be  repeated  two  weeks  later,  then  once  a month  until  the  end  of 
the  demonstration.  The  interview  question  related  to  this  objective 
can  be  found  in  the  total  interview  package  contained  in  Appendix  K. 

Key  personnel  time  logs  will  be  reviewed  by  the  PRC 
staff  each  week  as  they  are  received  from  the  ESO.  That  information 
plus  the  interview  responses  will  determine  if  any  changes  in  task 
assignment  or  level  of  effort  are  warranted.  If  there  are,  the  appro- 
priate recommendations  would  be  made  by  the  PRC  staff  after 
consultation  with  key  site  personnel. 
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When  approval  of  a change  is  necessary,  the  PRC  staff 
will  submit  the  recommendations  in  writing  to  the  Communications 
Officer . 


c.  Evaluation 

At  the  end  of  the  demonstration  in  February  1978,  the 
time  log  data  and  interview  responses  will  be  studied  by  PRC  staff  to 
assess  the  support  personnel  required  per  student.  The  extrapolation 
will  infer  the  personnel  requirements  for  an  operational  CMI  onsite 
system.  Conclusions  will  be  developed  by  April  7. 

5.  Personnel  Training  Requirements 

* 

This  objective  is  concerned  with  determining  the  training 
needed  for  site  staff  supporting  CMI  courses. 

a.  Preparation 

During  the  demonstration,  the  LS  will  receive  IT-BE/E 
training,  and  together  with  the  CMI  advisor,  will  train  the  ETs  to  car- 
ry out  their  responsibilities.  Time  spent  by  the  LS  and  CMI  advisor 
with  the  ETs  will  count  towards  training.  To  gather  this  data,  in  the 
preparation  stage  the  LS  and  CMI  advisor  will  maintain  logs  from  their 
time  of  arrival  at  Stockton.  Under  "Administrative  Activities"  they 
will  record  the  time  spent  and  the  particular  activity  in  training  the 
others,  including  all  staff  orientation  sessions  and  walkthroughs. 

These  LS  and  CMI  advisor  logs  will  be  accumulated  up  to  the  start  of 
the  demonstration. 

b.  Demonstration 

During  the  demonstration,  the  LS  and  CMI  advisor  will 
maintain  their  time  logs. 
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The  observation  of  procedures  by  the  PRC  staff  and  the 
periodic  interviews  to  assess  the  effectiveness  of  operations  (Appendix 
K)  will  address  the  adequacy  of  the  training.  The  schedule  of  observa- 
tion and  interviews  described  for  identifying  personnel  requirements 
will  be  followed  here. 

c.  Evaluation 

Beginning  March  13,  after  the  end  of  the  demonstration 
and  in  parallel  with  the  assessment  of  personnel  requirements,  the  data 
and  information  on  training  will  be  studied  and  the  actual  amount  of 
training  time  for  key  personnel  summarized.  The  interviews  will  iden- 
tify problems  of  inadequate  training  as  well  as  areas  of  satisfaction. 
An  assessment  will  be  made  of  the  adequacy  of  an  advisory  activity  dur- 
ing the  demonstration  versus  more  complete  preparatory  training. 
Recommendations  will  be  made  on  the  amount  of  training  for  each  key 
site  person,  the  subjects  to  be  covered,  and  the  schedule  of  that 
training.  This  evaluation  will  be  completed  by  April  7. 

6.  Organization  and  Management 

This  objective  is  concerned  with  determining  the  organization 
and  management  structure  needed  for  successful  operation  of  onsite  CMI 
courses. 


a.  Preparation 

The  workshop  of  August  29  will  finalize  the  organization 
of  the  demonstration.  The  interview  package  of  Appendix  K has  been  de- 
veloped to  address  the  capabilities  and  deficiencies  of  the  organi- 
zation and  management  structure,  and  how  the  CMI  operation  can 
interfere  with  routine  duties. 
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b.  Demonstration 

Organization  and  management  functions  will  be  observed 
by  PRC  staff  continuously  over  the  first  two  weeks  and  for  several  days 
each  month  thereafter.  The  first  major  interview  using  the  questions 
from  Appendix  K will  be  given  at  the  end  of  the  second  week.  Addition- 
al interviews  will  be  given  at  the  end  of  the  first  month  of  the  demon- 
stration and  once  a month  after  that.  The  interviews  will  precede  each 
progress  review  meeting. 

c.  Evaluation 

The  results  from  interviews  and  PRC  staff  observations 
will  be  studied  beginning  April  10.  From  the  interview  responses,  the 
organization  and  management  structures  used  during  the  demonstration 
will  be  described  and  subjectively  evaluated  for  effectiveness.  The 
evaluation  will  be  completed  by  April  28. 

7.  Space  Requirements  and  Operational  Procedures 

This  objective  is  concerned  with  the  extent  to  which  the  fa- 
cilities and  day-to-day  student  and  CMI  staff  procedures  enhance  or 
hinder  the  student's  progress. 

a.  Preparation 

The  facilities  will  be  in  place  by  August  20,  with  two 
storage  cabinets  and  an  LS  desk  provided  in  the  Building  120  conference 
room.  By  August  29,  the  learning  center  will  have  all  the  lab  equip- 
ment, microfiche  readers,  and  course  print  materials,  and  the  Opscan/ 
Terminet  cluster  connecting  to  the  CMI  computer  in  Memphis  will  be  in 
place. 

b.  Demonstration 

In  the  periodic  interviews  to  be  held  by  the  PRC  staff, 
the  ESO,  ETs,  LS,  CMI  advisor,  and  students  will  be  asked  to  assess  the 
adequacy  of  the  procedure  and  facilities. 
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c.  Evaluation 

A description  of  the  facility  setup  and  operational  pro- 
cedures followed  throughout  the  demonstration  will  be  prepared  by  PRC. 
The  interview  responses  will  be  reviewed  by  PRC  staff  and  an  assessment 
made  of  which  characteristics  are  important  for  successful  operation  of 
CMI  onsite.  Inferences  will  be  drawn  for  large  scale  operation  of 
CMI . This  evaluation  work  will  be  initiated  May  1 and  completed  by 
May  19. 


8 . Equipment,  Maintenance,  Spare  Parts,  and  Logistics 

Requirements 

This  objective  is  concerned  with  smooth  and  effective  opera- 
tion, and  includes  requirements  for  pieces  of  equipment,  maintenance 
response  time,  availability  of  spare  parts,  and  replenishment  of  stock 
and  supplies.  The  costs  of  these  requirements  are  considered  in  the 
economic  analysis.  The  following  paragraphs  emphasize  the  accept- 
ability of  the  operation  to  the  personnel. 

a.  Preparation 

The  equipment  and  spares  required  for  the  demonstration 
will  be  in  place  by  August  29.  Maintenance  for  the  communication  line 
and  the  Opscan/Terminet  cluster  will  be  provided  under  contract  while 
ETs  at  Stockton  will  maintain  the  lab  equipment  as  part  of  their  watch 
duties.  The  microfiche  readers  will  be  maintained  by  the  local  distri- 
butor at  Stockton.  Finally,  the  ESO  will  order  expendables  as  needed, 
following  the  established  routine  at  Stockton. 

b.  Demonstration 

The  observations  and  periodic  interviews  conducted  by 
PRC  staff  address  the  issue  of  personnel  satisfaction  with  the  demons- 
tration plan  for  equipment,  maintenance,  spare  parts,  and  logistics. 

In  addition,  data  will  be  collected  from  the  maintenance  logs  and  key 
personnel  time  logs  each  week  by  the  PRC  staff. 


TAEG  Report  No.  49 


c.  Evaluation 

The  PRC  staff  will  document  the  arrangements  for  equip- 
ment, maintenance,  spares,  and  logistics.  Performance  measures  will  be 
summarized  in  terms  of  the  maintenance  response  time,  down  time,  use  or 
lack  of  use  of  spares,  spares  shortages,  and  shortages  of  expendables. 

Interview  and  observation  results  will  be  used  to  iden- 
tify the  level  of  satisfaction  or  dissatisfaction  with  the  demonstra- 
tion procedures.  Requirements  for  onsite  delivery  of  CMI  will  be 
stated,  and  an  assessment  made  of  how  they  might  change  as  the  scale  of 
the  CMI  effort  changes.  The  evaluation  will  begin  May  22  and  be  com- 
pleted by  June  9. 


D.  Support  Functions 

This  section  describes  the  communications  system,  lab  equipment, 
facilities,  and  print  and  other  support  materials  required  to  carry  out 
the  demonstration. 


1.  Communications  System 


a.  System  Configuration 


Servicing  the  demonstration  communications  needs  re- 
quires an  unconditioned  voice  grade  telephone  line  connecting  the 
Opscan/Terminet  cluster  and  the  San  Diego  CMI  concentrator.  Three  land 
line  alternatives  were  considered  (as  described  in  Appendix  M)  and  a 
dedicated  commercial  line  was  selected  for  the  demonstration.  This  al- 
ternative is  expected  to  cost  on  the  order  of  $500  and  $550  more, 
respectively,  than  the  other  two  dial  line  alternatives  considered. 

The  latter,  however,  would  require  "extensive"  software  modifications 
according  to  Honeywell  and  Pacific  Telephone.  No  firm  cost  estimates 


were  available,  but  the  work  would  easily  cost  $1,500,  eliminating  the 
advantages  of  Alternatives  2 and  3 over  the  dedicated  commercial  line. 
In  addition,  the  time  and  effort  required  to  revise  Honeywell's  con- 
tract for  the  CMI  concentrator  would  be  eliminated. 
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Details  of  the  cx>inparison  of  these  three  alternatives, 
including  the  cost  considerations,  are  contained  in  Appendix  M. 


b. 


Opscan/Terminet  Cluster 
The  CMI  system  requires  an  Opscan  17  optical  reader  and 
GE  Terminet  1200  teletype  terminal.  As  described  earlier,  student  test 
sheets  are  fed  into  the  Opscan  and  the  information  is  transmitted  to 
the  CMI  computer  in  Memphis  via  the  concentrator  at  San  Diego.  After 
the  test  is  graded,  instructions  to  the  student  are  printed  out  on  the 
Terminet. 


A cluster  consisting  of  one  Opscan  and  one  Terminet  will 
be  placed  in  the  learning  center.  A second  cluster  will  be  kept  in  the 
ET  storeroom  in  the  blockhouse  as  a spare.  The  Opscan  and  Terminet 
will  each  be  on  a table  with  castors  so  they  can  be  rolled  into  place 
quickly  when  needed. 


Maintenance  of  the  Opscan  and  Terminet  equipment  will  be 
provided  by  the  respective  contractors.  National  Computer  Services 
(NCS)  will  maintain  the  Opscan,  and  Honeywell  will  have  the  responsi- 
bility for  the  Terminet.  In  each  case,  service  will  be  on  call.  The 
availability  of  a single  spare  for  each  component  of  the  cluster  is 
considered  more  than  adequate  based  or.  MIISA  experience. 


c.  Preparation 

By  July  1,  1977,  CNET  will  have  tasked  the  CMI  System 
Manager  to  task  MIISA  Memphis  to  do  the  following: 


Lease,  via  OOMNAVTELCOM , a two  pair,  type  3002,  full  duplex, 
unoonditioned  line  from  Stockton  to  San  Diego.  Bell  202-T 
modems  should  be  used  at  eaoh  end  of  the  line.  The  line  will 
be  activated  August  29,  1977,  at  the  latest. 


Have  Honeywell  Install  the  RP6352  asynchronous  line  modulator 
at  San  Diego  by  August  29,  1977. 
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• Provide  Stockton  with  two  installed  and  operating  OPSCAN/ 
Terminet  clusters  by  August  29,  1977.  Delivery  will  take 
place  by  August  20,  1977. 


By  July  15,  MIISA  Memphis  will  complete  contract  nego- 
tiations with  Honeywell  for  the  following: 

• Delivery  of  two  Opscan/Terminet  clusters  to  Stockton  by 
August  20 

• Installation  and  checkout  of  one  cluster  at  Stockton  by 
August  29 

• Maintenance  of  the  cluster  on-call  by  Honeywell  for  the 
Terminet,  NCS  for  the  Opscan 

• Installation  of  the  RP6352  asynchronous  line  modulator  on  the 
CMI  concentrator  at  San  Diego  by  August  29 

By  the  same  date,  MIISA  must  initiate  negotiations 
through  COMNAVTELCOM  for  a contract  with  Pacific  Telephone  to  provide  a 
dedicated  phone  line  between  Stockton  and  San  Diego. 

On  August  1 the  Opscan/Terminet  clusters  by  Honeywell 
and  NCS  will  be  shipped.  August  20  is  the  latest  date  for  delivery  of 
the  clusters. 


On  August  29th  Pacific  Telephone  will  have  an  operating 
dedicated  line,  including  modems,  between  Stockton  and  San  Diego. 
Honeywell  will  have  completed  installation  of  the  asynchronous  line 
modulator  to  tie  in  the  CMI  concentrator  in  San  Diego.  Honeywell  and 
NCS  will  also  have  completed  installation  and  checkout  of  the  cluster 
so  there  will  be  a fully  operational  CMI  network  on  that  date. 

d.  Demonstration 

Maintenance  will  be  provided  by  Pacific  Telephone  for 
the  dedicated  line  and  modems,  by  Honeywell  for  the  RP6352  asynchronous 


TABG  Report  No.  49 


r' 


v* 

•X 

i > 

I ? 

I V 
' V 

’ V 

! 

i 


I 


line  modulator  and  Termlnet  on  call,  and  by  NCS  for  the  Opscan.  The 
system  will  be  used  routinely  beginning  September  6 and  will  be  closed 
down  by  February  28,  1978. 

The  maintenance  log  will  be  kept  up  to  date  during  the 
demonstration  for  the  Opscan,  Termlnet,  and  the  dedicated  line  with  mo- 
dems and  the  line  modulator. 

The  Opscan/Termlnet  cluster  will  be  disconnected  by 
Honeywell  and  NCS  on  February  28  and  the  units  shipped  to  a site  desig- 
nated by  MIISA.  On  the  28th  the  dedicated  phone  line  will  be  discon- 
nected by  Pacific  Telephone. 

e.  Evaluation 

In  the  economic  analysis,  cost,  down  time,  ard  mainten- 
ance data  obtained  from  the  demonstration  will  be  used  to  assess  the 
cost  effectiveness  of  delivering  CMI  to  remote  sites.  That  study  will 
be  finished  June  1.  The  effectiveness  of  the  maintenance  program  and 
the  reliability  of  the  system  are  of  interest  in  the  analysis  of  equip- 
ment, maintenance,  spares,  and  logistics  requirements  to  be  completed 
J une  9 . 


2.  Laboratory  Equipment  and  Microfiche  Readers 

Table  IV-1  lists  the  laboratory  equipment  and  microfiche 
readers  needed  for  Course  69 . The  cost  of  the  equipment  totals  about 
$6,200. 


The  equipment  needed  for  the  demonstration  is  standard  for 
Course  69  except  that  testing  versions  of  the  NEAT  boards  have  to  be 
modified.  A lock  box  will  be  added  over  the  test  control  knobs  because 
the  boards  will  be  used  for  practice  as  well  as  testing  to  reduce  the 
amount  of  equipment  needed.  Since  it  would  be  possible  for  students  to 
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I'  Table  / 1.  Equipment  List  for  BE/E  Course  69 

S 


r 


decipher  the  variable  settings  of  the  test  knobs  during  practice,  secu- 
rity has  to  be  provided  in  the  form  of  a lock  box.  The  LS  and  ETs  will 
keep  the  keys. 


Itemi 

Unit 

Price 

Total 

1. 

Four  Simpson  260-5P  Multimeter 

$138.00 

$ 

562.00 

2. 

Three  RCA  Oscilloscope  Type  WO-33A 

$275.00 

$ 

825.00 

3 

Three  EICO  Model  337  Signal  Generator 

$120.00 

$ 

160.00 

4 

Three  RCA  WV-77E  VTMV 

$ 69.00 

$ 

207.00 

5. 

Three  WG-349A  Direct/ Low-Capacitance  Probe  and  Cable 

$ 15.00 

$ 

45.00 

6. 

Training  Devices 

Two  N.E.A.T.  Training  Device  11 

$300.00 

$ 

600.00 

Two  N.E.A.T.  Training  Device  /fla 

$300.00 

$ 

600.00 

Two  N.E.A.T.  Training  Device  f2a 

$400.00 

$ 

800.00 

Two  N.E.A.T.  Training  Device  #6 

$300.00 

$ 

600.00 

Two  N.E.A.T.  Training  Device  f? 

$300.00 

$ 

600.00 

Two  N.E.A.T.  Training  Device  #9 

$300.00 

$ 

600.00 

7. 

Twelve  Dry  Cells,  1 .5v 

$ 3.00 

$ 

36.00 

8. 

Four  Hook-Up  Wires  lOff  the  shelfl 

$ 2.00 

$ 

8.00 

9. 

Four  Sets  of  Meter  Leads  lOff  the  shelfl 

$ 4.00 

$ 

16.00 

10. 

Two  Microfiche  Readers 

$175.00 

$ 

350.00 

TOTAL 

$6,'lS9.TO 

a.  Preparation 

By  July  1,  CNET  will  have  tasked  CNTECHTRA  to  provide: 

• Two  full  sets  of  NEAT  testing  boards  on  a no-cost  loan,  modi- 
fied with  a lock  box  covering  the  test  control  knobs 

• The  type  and  number  of  items  of  Course  69  lab  equipment 
needed  for  50  students 
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This  will  be  shipped  by  August  1 and  delivered  to  Stockton  by  August  20. 

The  liS  and  an  ET  will  check  out  the  lab  equipment  and 
microfiche  readers  on  August  25.  Repairs  or  replacements  will  be  ar- 
ranged as  needed. 

b.  Demonstration 

The  lab  equipment  will  be  used  by  students  under  the 
supervision  of  the  LS,  or  an  BT  when  the  LS  is  off  duty.  The  micro- 
fiche readers  will  be  used  for  CMI  tests. 

Maintenance  of  this  equipment  will  be  provided  by  ETs. 
There  are  no  data  available  on  maintenance  requirements,  but  the  staff 
using  the  equipment  at  CNTECRTRA  Memphis  see  no  need  for  major  effort. 

An  estimate  of  16  hours  per  month  has  been  made  by  PRC  to  cover  main- 
tenance. 


A maintenance  log  will  be  kept.  The  microfiche  readers 
will  be  serviced  by  a local  distributor. 

The  LS  and  ETs  will  keep  a record  of  equipment  use  to 
identify  whether  or  not  the  provided  equipment  is  adequate  and 
necessary. 


c.  Evaluation 

The  cost  and  frequency  of  repair  data  will  be  used  in 
the  economic  analysis.  Equipment  and  maintenance  needs  for  onsite  CMI 
will  be  assessed  based  on  the  use  data  and  results  from  interviews  and 
observation.  The  study  will  be  completed  June  1. 

3.  Pacilities 

The  learning  center  will  be  located  in  the  blockhouse 
(Building  120)  conference  room.  Two  large  conference  tables,  arranged 
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in  a ”T”  shape,  will  remain.  Space  is  available  at  one  end  of  the  roon 
for  the  Opscan/Termlnet  cluster,  two  standard  six  foot  high  metal  stor- 
age cabinets,  and  a small  desk  for  the  LS.  The  cabinets,  trtiich  will 
contain  lab  equipment,  study  materials,  and  microfiche  readers  will  be 
locked  when  not  in  use.  Figure  III-7,  in  Chapter  III  on  evaluating 
facilities,  shows  the  dimensions  of  the  room  and  possible  placoient  of 
the  equipment. 

A maximum  of  five  trainees  is  expected  at  any  one  time  for 
scheduled  study,  testing,  or  laboratory  work.  Others  may  come  in  on 
their  free  time  to  move  ahead  in  the  work  or  do  remedial  work.  The 
trainees  will  work  at  the  conference  tables. 

Staff  meetings  will  have  priority  over  COMISAT  for  use  of  the 
conference  room.  In  order  to  minimize  disruption  of  course  work,  an 
effort  will  be  made  at  Stockton  to  schedule  staff  meetings  several  days 
ahead.  This  would  give  the  LS  enough  time  to  adjust  trainee  schedules 
for  working  in  the  center. 

a.  Preparation 

By  July  11,  the  Stockton  ESO  will  also  verify  that  t*ro 
storage  cabinets  and  a desk  can  be  provided  for  the  learning  center. 

By  August  20,  the  ESO  will  have  equipped  the  learning  center  with  stor- 
age cabinets  for  course  materials  and  equipment,  and  a desk  for  the 
LS.  As  soon  as  the  materials  and  equipment  arrive,  the  ESO  will  have 
them  stored  in  the  learning  center. 

On  August  29,  a workshop  dealing  with  the  schedules  and 
tasks  of  the  personnel  taking  part  in  the  demonstration  will  also  ad- 
dress the  scheduling  of  the  learning  center.  Provision  will  be  made 
for  Fleet  Center  staff  conferences.  It  is  expected  that  there  will  be 
little  variation  from  the  agreed  upon  schedule. 
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b.  Demonstration 

The  learning  center  will  remain  open  24  hours  a day. 
However,  when  Fleet  Center  staff  conferences  are  held  all  CMI  activi- 
ties will  stop  and  CMI  personnel  will  leave  the  conference  room. 

During  the  week,  the  LS  will  occupy  the  desk  In  the  room 
from  0000  to  1600  hours.  The  rest  of  the  time  the  ETs  will  be  on  call 
to  come  In  and  provide  assistance. 

The  observations  and  periodic  Interviews  of  students  and 
support  personnel  by  PRC  staff  will  provide  data  on  the  adequacy  of  the 
facility  for  the  CMI  course.  By  February  28,  the  equipment  and  ma- 
terials will  have  been  returned  to  CNTECHTRA,  the  Opscan/Termlnet  clus- 
ter shipped  to  MIISA,  the  phone  line  disconnected,  and  the  storage 
cabinets  and  LS  desk  removed. 

c.  Evaluation 

The  assessment  of  facility  needs  will  consider  user 
satisfaction  based  on  the  Interview  results  and  observation  by  PRC 
staff.  The  adequacy  of  the  space  and  organization  of  the  room  will  be 
assessed.  Inferences  will  be  drawn  on  the  requirements  for  an  opera- 
tional CMI  system  in  the  evaluation  study  to  be  completed  May  19. 

4.  Print  and  Other  Materials 

The  following  materials  will  be  provided  by  the  agency  Indi- 
cated in  parentheses: 

• Ten  copies  of  BE/E  Course  69  core  module  (1-14)  texts 
(CNTECHTRA) 

• Short  and  long  Opscan  test  answer  sheets,  lab  sheets, 
administrative  forms  (PI,  P2,  P3),  Terminet  paper  (CNTECHTRA) 

• Student  and  Support  personnel  log  sheets  (PRC) 

• Expendable  supplies  such  as  pens,  pencils,  paper  (NAVCOMMSTA 
Stockton) 
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In  addition,  the  ESO  will  order  correspondence  course  material  from  the 
Naval  Education  and  Training  Program  Development  Center  in  Pensacola. 

a.  Preparation 

By  July  If  CNET  will  have  tasked  CNTECHTRA  to  provide  10 
sets  of  Course  69  texts,  modules  1-14,  and  enough  test  sheets,  micro- 
fiche tests,  lab  papers,  Opscan  administrative  forms  (PI,  P2,  P3) , and 
Terminet  paper  to  support  the  experiment.  These  will  be  delivered  by 
August  20,  1977,  and  stored  in  the  learning  center. 

By  July  25  the  ESO  will  order  the  correspondence 
materials.  They  are  to  be  delivered  by  August  23. 

b.  Demonstration 

The  students  will  be  given  access  to  the  materials  by 
the  LS,  or,  when  the  LS  is  off  duty,  by  the  ETs  substituting  for  the  LS. 

During  the  demonstration,  the  periodic  interviews  and 


observations  by  PRC  staff  will  obtain  information  on  any  bottlenecks 
that  may  exist  with  respect  to  materials. 

c.  Evaluation 

A judgment  will  be  made  by  the  PRC  staff  on  whether  or 
not  the  materials  were  adequate  in  number  and  availability.  Implica- 
tions might  be  drawn  on  requirements  for  an  operational  CMl  system. 
However,  the  test  case  is  small  for  this  subject  and  it  is  doubtful 
strong  conclusions  can  be  reached.  The  analysis  will  be  completed  by 
June  9. 
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Chapter  V 
TASKING 


A.  Introduction 

A successful  demonstration  will  require  coordination  and  support 
from  a number  of  commands  and  operating  units  within  the  Navy.  This 
chapter  describes  the  specific  tasking  required  within  the  Navy  to  car- 
ry out  the  demonstration. 

B.  Organizational  Responsibilities 

Within  the  Navy  there  are  four  groups  involved  in  the  tasking: 
CNET,  OOMNAVTELOOM,  CNTECHTRA  and  MI ISA. 

CNET  is  responsible  for  preparing  and  issuing  the  tasking.  The 
tasking  for  the  communication  lines  has  to  be  approved  by  OP-94  at 
COMNAVTELCOM. 

CONNAVTELOOM  will  be  responsible  for  approving  the  use  of  commer- 
cial leased  lines  to  connect  Stockton  with  San  Diego,  including  the  se- 
lection of  modems  to  connect  the  lines  with  the  terminal  equipment.  In 
gaining  this  approval,  justification  for  the  use  of  commercial  lines 
instead  of  available  Navy  AOTODIN  lines  must  be  shown.  The  arguments 
presented  in  the  preceding  chapter  and  Appendix  M on  the  selection  of 
the  coamiunlcatlon  line  provide  the  basis  for  this  justification. 

CNTECHTRA  will  be  responsible  for  providing  a learning  supervisor, 
the  BE/E  Course  Pile  69  laboratory  equipment,  and  the  textbooks,  test 
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forms,  answer  sheets,  and  other  materials  related  to  the  course.  The 
costs  associated  with  these  Items  are  to  be  covered  by  CNET. 

MUSA  will  be  responsible  for  providing  specifications  on  the  com- 
munication lines  and  modems,  ordering  and  delivering  the  OPSCAN/ 
TERMINET  clusters,  and  providing  student  data  both  in  preparation  for 
and  during  the  demonstration.  Because  of  a change  in  ownership  of  the 
manufacturer  of  the  OPSCAN  equipment,  MIISA  will  have  to  work  out  an 
agreement  for  the  additional  cluster.  These  negotiations  may  be 
lengthy  and  require  an  early  start  in  order  for  the  equipment  to  be 
ready  for  the  demonstration. 

Three  types  of  student  data  are  required  from  MIISA  in  order  to 
efficiently  carry  out  the  demonstration.  The  first  type  consists  of 
projections  of  time  to  completion  for  students  with  the  skill  levels 
representative  of  the  volunteers  at  Stockton.  This  is  to  be  obtained 
by  enrolling  the  volunteers,  if  they  are  known,  with  names  and  social 
security  numbers  masked,  or  a fictitious  set  of  students  with  represen- 
tative aptitude  levels.  The  immediate  output  of  the  CMI  computer  will 
provide  an  estimated  time  to  completion.  Once  that  is  obtained,  the 
enrollments,  fictitious  or  actual,  will  be  erased  from  the  computer 
files  pending  enrollment  of  the  final  set  of  students  at  the  beginning 
of  the  demonstration.  The  second  type  of  data  required  is  routine 
progress  reports  for  the  students  actually  enrolled  in  the  demonstra- 
tion. As  indicated  earlier  these  will  be  mailed  by  MIISA  San  Diego, 
initially  twice  a week  and  then  on  a weekly  basis,  to  the  learning 
supervisor  at  Stockton.  The  third  set  of  data  will  contain  information 
on  the  learning  effectiveness  control  group  and  will  be  derived  from 
historical  data  contained  in  the  CMI  computer  at  Memphis.  Two  steps 
are  required  to  obtain  that  data.  First,  files  will  be  searched  to 
identify  graduates  of  the  BE/E  Course  File  69  with  paygrade  levels  in 
the  B4  to  E6  group.  Once  a control  group  has  been  identified,  per- 
formance data  on  time  to  completion  will  be  obtained. 
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In  addition  to  student  data,  MIISA  personnel  are  to  provide  PRC 
with  data  on  personnel  time  expended  for  the  COMISAT  project. 

C.  Tasking  Letter 

The  tasking  letter  to  be  issued  by  CNET  will: 

• Set  deadlines 

• Initiate  long  lead-time  activities 

• Where  additional  work  is  required  before  requirements  are 
completely  known,  allow  resolution  of  details  between  the  ap- 
propriate Navy  group  and  the  project  staff  at  PRC 

In  addition,  to  ensure  proper  coordination  of  all  activities,  the  let- 
ter should  clearly  specify  communication  links  between  the  Navy  groups 
participating  in  the  demonstration,  the  PRC  project  staff,  and  CNET. 


The  tasking  letters  should  detail  the  following  activities  to  be 
carried  out  by  each  organizatic«i: 

• MIISA 

MIISA  San  Diego  to  register  the  students,  and  prepare 
and  mail  student  reports  twice  a week  for  first  month, 
weekly  thereafter 

MIISA  Memphis  to  establish  a Complex  Identification  Num- 
ber for  the  Stockton  trainees;  estimate  time  to  comple- 
tion for  a group  of  trainees  representative  of  Stockton 
based  on  data  supplied  by  PRC 

All  involved  sections  of  MIISA  to  submit  all  costs  and 
time  charged  to  the  project  in  a form  suitable  for 
analysis  as  specified  by  PRC 

• CNTBCHTRA 

— CMI  System  Manager  to  task  MIISA  to  provide  Stockton 
with  two  installed  and  operating  OPSCAN/Terminet  clus- 
ters by  August  29,  1977.  Delivery  will  take  place  by 
August  20,  1977 
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cm  System  Manager  to  task  MIISA  to  have  Honeywell  in- 
stall the  RP63S2  asynchronous  line  modulator  at  San 
Diego  by  August  29,  1977 

To  provide  10  sets  of  Course  69  texts,  modules  1-14,  and 
enough  test  sheets,  microfiche  tests,  lab  papers,  Opscan 
administrative  forms  (PI,  P2,  P3)  and  Terminet  paper  to 
support  the  experiment.  These  will  be  delivered  by 
August  20,  1977 

To  provide  two  full  sets  of  NEAT  testing  boards  on  a 
no-cost  loan,  modified  with  a lock  box  covering  the  test 
control  knobs;  to  provide  the  type  and  number  of  items 
of  Course  69  lab  equipment  needed  for  50  students.  This 
will  be  delivered  to  Stockton  by  August  20,  1977 

To  train  and  assign  an  LS  to  arrive  at  Stockton  by 
August  20,  1977.  The  IT-BE/E  training  program  will  be- 
gin July  18,  1977.  If  necessary,  more  than  one  LS  will 
be  used  in  sequence  to  maintain  a full  time  position 

To  identify  and  maintain  files  on  a control  group  for 
attitudes 

BE/E  TPC  to  assist  in  identifying  a group  of  55  trainees 
to  validate  the  attitude  test  at  Great  Lakes  Training 
Center,  55  at  Memphis.  The  contact  at  Great  Lakes  would 
be  identified  and  all  arrangements  made  by  the  TPC. 

This  will  begin  by  July  5,  1977 

To  assign  a staff  member  to  oversee  the  setup  of  opera- 
tions at  Stockton.  Dates  to  be  specified 

To  submit  all  costs  and  time  charged  to  the  project  in  a 
form  suitable  for  analysis  as  specified  by  PRC 


A sample  translation  of  these  requir«inents  into  a tasking  letter 
is  provided  in  Appendix  N. 


Oil  System  Manager  to  task  MIISA  to  lease,  via 
COMNAVTELCOM,  a two  pair,  type  3002,  full  duplex,  uncon- 
ditioned line  from  Stockton  to  San  Diego.  Bell  202-T 
modems  should  be  used  at  each  end  of  the  line.  The  line 
VI  will  be  activated  August  29,  1977,  at  the  latest 
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•X 

Chapter  VI 

DH40NSTRATI0N  MASTER  PLAN 

! 

V 

^ This  chapter  chronologically  describes  the  preparation,  conduct, 

• and  evaluation  of  the  demonstration  from  completion  of  this  report  to 

evaluation  of  the  results  of  the  demonstration.  Figure  VI-1  is  a flow 
chart  of  the  activities. 

^ A.  Preparation 

t 

Upon  completion  of  the  demonstration  design,  CNET  will  provide 
tasking  for  actions  to  be  taken  in  preparation  for  the  demonstration. 
These  tasks  are  described  in  Chapter  IV. 

By  July  11,  the  ESO  at  Stockton  will  provide  PRC  with  a represen- 
tative list  of  student  characteristics  based  on  the  Chief's  survey  of 
potential  volunteers  for  the  CMI  course.  By  July  15,  MIISA  Memphis 
will  register  these  students  in  the  BE/E  Course  File  69  to  predict  com- 
pletion times.  The  students  will  then  be  dropped  from  the  file  and  the 
results  given  to  PRC  staff  for  use  in  finalizing  the  demonstration 
schedule.  The  PRC  staff  will  be  at  Memphis  when  the  data  are  obtained. 

Also  by  July  11,  the  ESO  at  Stockton  will  verify  that  two  storage 
cabinets  and  a desk  can  be  provided  for  the  learning  center.  By  July 
15,  the  BE/E  TPC  at  CNTECHTRA  will  assign  an  individual  who  is  familiar 
with  the  BE/E  Course  File  69  as  a CMI  advisor  to  assist  in  setting  up 
the  demonstration  at  Stockton. 
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An  LS  will  be  identified  by  CNTECHTRA  by  July  11.  Also  by  July 
11,  the  BE/E  TPC  will  arrange  for  the  LS  to  begin  any  necessary  IT-BE/E 
training  on  July  18. 

By  July  15,  MUSA  Memphis  will  complete  contract  negotiations  with 
Honeywell  for  the  following; 

• Delivery  of  two  Opscan/Terminct  clusters  to  Stockton  by 
August  20 

• Installation  and  checkout  of  one  cluster  at  Stockton  by 
August  29 

• Maintenance  of  the  cluster  on-call  by  Honeywell  for  the  Ter- 
mlnet,  by  NCS  for  the  Opscan 

• Installation  of  the  RP6352  asynchronous  line  modulator  on  the 
CMl  concentrator  at  San  Diego  by  August  29 

By  the  same  date,  MlISA  must  initiate  negotiations  through  OOMNAVTEL- 
COM  for  a contract  with  Pacific  Telephone  to  provide  a dedicated  phone 
line  between  Stockton  and  San  Diego. 

The  training  of  the  LS  at  the  training  center  in  Memphis  or  San 
Diego  BE/E  school  will  begin  July  18  and  finish  August  5.  On  the  same 
day,  PRC  staff  will  begin  a week-long  validation  of  the  attitude  test 
with  55  students  at  the  Great  Lakes  training  center. 

On  July  18,  MUSA  Memphis  will  begin  a file  search  of  Course  File 
69  records  to  identify  an  historical  set  of  control  data  for  students 
with  similar  BTB  scores  and  paygrades  from  E4  to  E6.  Upon  completion 
of  this  search,  PRC  staff  will  screen  the  results  and  select  a control 
group  by  July  29. 

Beginning  July  18,  the  ESO  at  Stockton  will  survey  offsite  person- 
nel to  determine  the  number  who  will  take  the  BE/E  correspondence 
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course.  On  July  25,  the  ESO  will  order  the  necessary  test  and  test  ma- 
terials from  the  Naval  Education  and  Training  Program  Development  Cen- 
ter in  Pensacola. 

During  the  week  of  July  25-29,  PRC  staff  will  administer  the  sec- 
ond stage  of  the  attitude  test  validation  to  55  CMI  students  at  the 
Memphis  training  center. 

On  August  1,  there  will  be  two  shipments:  (1)  the  Opscan/Terminet 
clusters  by  Honeywell  and  NCS,  and  (2)  Course  69  print  material,  lab 
equipment,  and  microfiche  readers  by  CNTECHTRA.  Also  during  this  week 
the  ESO  at  Stockton  will  finalize  the  list  of  volunteers  for  both  the 
CMI  course  and  the  correspondence  course. 

On  August  8,  with  student  lists  complete,  the  ESO  at  Stockton  will 
prepare  the  needed  registration  information  and  provide  copies  to  San 
Diego  and  the  PRC  staff  who  will  be  in  Stockton  August  8 through  Sep- 
tember 16 . 

Also  on  August  8,  MIISA  Memphis  will  establish  a Complex  Identifi- 
cation Number  for  the  CMI  learning  center  at  Stockton  and  notify  MIISA 
San  Diego  of  the  file  number  so  the  latter  can  handle  administrative 
actions  on  the  computer  for  Stockton. 

On  August  12,  MIISA  San  Diego  will  register  the  Stockton  CMI  stu- 
dents and  mall  copies  of  the  initial  roster  to  the  ESO  at  Stockton  and 
to  PRC  in  McLean,  Virginia.  The  roster  should  be  received  at  Stockton 
on  Monday,  August  15,  at  which  time  the  ESO  and  the  PRC  staff  at  Stock- 
ton  will  schedule  student  course  activities  based  on  the  estimated 
course  completion  time  contained  in  the  roster.  A tentative  schedule 
will  be  prepared  by  August  19,  and  revised  in  a planning  session  sched- 
uled for  August  29. 
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August  20  is  the  last  day  for  arrival  of  the  LS  and  the  CKI  advi- 
sor and  delivery  of  the  Opscan/Terminet  clusters  and  the  course  mater- 
ials and  equipment.  By  then  the  BSO  will  have  equif^d  the  learning 
center  with  storage  cabinets  for  course  materials  and  equipment,  and  a 
desk  for  the  LS.  As  soon  as  the  materials  and  equipment  arrive,  the 
ESO  will  have  them  stored  in  the  learning  center. 

The  correspondence  course  materials  are  expected  by  August  23.  On 
August  25,  the  LS  and  an  ET  will  check  the  lab  equipment  and  microfiche 
and  arrange  repair  or  replacements  as  needed. 

By  August  26,  the  PRC  staff  and  the  ESO  will  administer  the  atti- 
tude pretest  and  interview  to  the  CMI  students  and  key  personnel. 

Beginning  the  afternoon  of  August  29  and  continuing  throughout  the 
week,  the  on-watch  orientation  meetings  will  be  held  with  the  ETs. 

These  meetings,  which  will  last  three  hours,  will  prepare  the  ETs  for 
their  roles  as  substitutes  for  the  LS  and  as  maintenance  personnel  for 
the  lab  equipment. 

On  August  29,  Pacific  Telephone  will  have  an  operating  dedicated 
line,  including  modems,  between  Stockton  and  San  Diego.  Honeywell  will 
have  completed  installation  of  the  asynchronous  line  modulator  to  tie 
in  the  CMI  concentrator  in  San  Diego.  Honeywell  and  NCS  will  also  have 
completed  Installation  and  checkout  of  the  cluster  so  there  will  be  a 
fully  operational  CMI  network  on  that  date. 

A workshop  will  be  held  on  the  morning  of  August  29  to  develop  a 
complete  scheduling  program  for  the  operation  of  the  CMI  course.  At 
this  workshop,  the  LS,  CMI  advisor,  PRC  staff.  Division  Officers,  Com- 
munications Officer,  ESO,  and  DTC/POs  from  Stockton  will  develop  a 
schedule  for  the  LS  and  ETs  over  the  expected  duration  of  the  course. 
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Beginning  the  afternoon  of  August  29  and  continuing  throughout  the 
week,  aeetings  will  be  held  with  the  ETs  to  set  student  schedules.  The 
PRC  staff,  the  LS,  the  CHI  advisor,  and  the  Division  Officers  will  par- 
ticipate. 

Beginning  the  afternoon  of  August  30,  and  extending  through  Sep- 
tember 2,  the  students  will  be  briefed  on  the  course.  The  students, 
ETs,  DTC/POs,  and  ESO  will  be  walked  through  the  operations  of  the 
cluster  and  the  use  of  the  materials  in  the  learning  center  by  the  LS 
and  CMI  advisors.  These  walkthroughs  will  be  scheduled  in  the  workshop 
of  August  29  and  coordinated  to  avoid  conflict  with  the  ET  meetings. 

B.  Demonstration 

Tuesday,  September  6 will  begin  the  demonstration.  The  LS  or  ET 
will  initiate  each  student  into  the  system  as  they  arrive.  For  the  du- 
ration of  the  course,  the  students  will  devote  an  average  of  two  hours 
per  watch  to  study,  lab  work,  and  exams.  The  last  student  is  expected 
to  finish  by  February  24.  when  the  course  begins  there  will  be  a re- 
vised estimate  of  the  end  date. 

The  correspondence  course  students  will  also  begin  on  September  6 
and  are  expected  to  finish  in  about  18  weeks,  assuming  an  average  of  10 
hours  per  watch  string.  Allowing  for  holidays,  they  will  finish  by 
January  20.  A revised  estimate  of  completion  will  be  made  by  the  PRC 
staff  for  the  August  30  workshop  using  time-to-completlon  estimates 
from  MUSA. 

The  CMI  course  will  operate  under  the  direction  of  the  LS,  with 
the  ETs  substituting  on  administrative  matters  and  technical  questions 
when  the  LS  is  not  on  duty.  The  KTs  will  also  maintain  the  lab  equip- 
ment. Student  progress  reports  in  the  form  of  learning  center  rosters 
will  be  provided  on  a weekly  basis  by  NIISA  San  Diego.  The  reports 


TAEG  Report  No.  49 


'I 


will  be  printed  and  mailed  Friday  at  San  Diego  for  receipt  by  the  LS  at 
Stockton  nn  Monday.  During  the  first  month,  an  additional  roster  will 
be  mailed  out  on  each  Tuesday. 

As  the  CMI  course  progresses,  adjustments  will  be  made  to  sched- 
ules and  procedures  for  students  and  ETs.  During  the  first  two  weeks, 
at  least  two  meetings  to  discuss  changes  will  be  chaired  by  the  LS. 

The  participants  will  be  the  LS,  CMI  advisor,  ESO,  DTC/POs,  Division 
Officers,  Ccnmiunications  Officer,  ETs,  and  PRC  staff.  PRC  staff  will 
observe  the  first  two  weeks  of  the  course  and  develop  an  agenda  for  the 
first  two  meetings,  plus  an  assessment  on  proposed  changes  to  be  dis- 
cussed in  the  meetings.  Improved  schedules  and  procedures  will  be 
generated.  After  the  first  two  weeks,  monthly  meetings  will  be  sched- 
uled to  coincide  with  visits  by  PRC  staff. 

In  order  to  gather  data  and  information  on  costs,  personnel  time, 
and  procedures,  support  personnel  at  Stockton  (the  LS,  CMI  advisor, 

ESO,  DTC/POs,  ETs,  Division  Officers,  and  Communication  Officer)  will 
maintain  logs  of  their  activities.  The  Division  Officers  and  Communi- 
cations Officer  will  note  the  time  spent  by  any  staff  not  keeping  a 
log.  These  logs  will  be  collected  each  Monday  by  the  ESO  and  forwarded 
to  PRC  at  McLean,  Virginia. 

MIISA  personnel  at  San  Diego  and  Memphis  will  also  report  time  and 
activities  devoted  to  the  project.  The  Computer  Center  Director  at 
MIISA  San  Diego  will  send  a weekly  report  to  PRC  at  McLean  each  Friday 
on  costs  and  personnel  time.  The  Deputy  Director  of  MIISA  at  Memphis 
will  report  any  significant  costs  or  time  spent  on  the  demonstration  in 
a weekly  report. 

In  addition  to  the  cost  and  time  data,  PRC  will  conduct  routine 
interviews  with  CMI  students  and  key  support  personnel  to  assess  how 
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well  the  demonstration  Is  functioning  In  terms  of  personnel  time,  per- 
sonnel training  needs,  organization  and  management  structure,  facili- 
ties, operational  procedures,  and  logistics.  These  interviews  will  be 
given  at  the  end  of  the  first  two  weeks,  the  first  month,  and  each 
month  thereafter  until  the  end  of  the  demonstration. 

The  students  in  the  correspondence  course,  which  will  run  concur- 
rently with  the  CMI  course,  will  also  work  an  average  of  10  hours  per 
watch  string.  There  will  be  little  formal  interaction  with  support 
personnel  in  the  correspondence  course  and  the  DTC/POs  at  the  off-base 
site  will  forward  student  tests  to  the  ESO,  who  will  review  them  and, 
at  the  end  of  the  course,  forward  them  to  Pensacola  for  grading.  The 
ESO  will  assist  the  DTC/POs  in  counselling  students  having  any  problems. 


Several  specific  events  will  occur  during  the  demonstration. 

• September  19  the  CMI  advisor  will  return  to  Memphis. 

• PRC  staff  and  the  Stockton  ESO  and  DTC/POs  will  administer 
the  Stanford  Diagnostic  Reading  Test  to  CMI  students  during 
the  week  of  October  3-7,  and  to  correspondence  students  dur- 
ing the  week  of  October  10-19.  The  results  will  be  used  in 
assessing  CMI  student  performance  in  the  BE/E  course  69  and 
both  groups  on  the  special  achievement. 

• Tests  and  interviews  to  evaluate  attitudes  will  be  adminis- 
tered to  a control  group  of  experienced  personel  attending  a 
CMI  course  from  a land  site.  The  dates  and  location  will  be 
set  by  PRC  and  CNTECHTRA,  with  completion  expected  by  Novem- 
ber 15.  Since  the  students  will  be  working  full-time  on  the 
course,  the  time  between  the  pretests  and  the  final  posttests 
may  be  as  little  as  six  weeks.  The  control  group  testing  and 
interviewing  is  expected  to  be  completed  by  the  end  of  Decem- 
ber. By  December  15,  the  special  exam  to  be  used  in  compar- 
ing the  effectiveness  of  the  CMI  and  correspondence  courses 
will  be  designed. 

• At  the  completion  of  the  correspondence  course  (expected  to 
end  by  January  20,  1978),  the  ESO  will  complete  the  student 
files  and  send  a summary  of  student  time-to-complete  and  to- 
tal performance  scores  to  PRC  at  McLean  by  January  27. 
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The  demonstration  will  be  closed  down  by  February  28,  1978.  On 
February  24  the  last  student  is  expected  to  log  out,  and  on  February  27 
MIISA  San  Diego  will  mail  out  the  Graduate  Performance  Summary  and  At- 
trition Student  Performance  Summary  to  PRC  at  McLean  and  the  LS  at 
Stockton.  The  ESO  will  collect  and  forward  to  PRC  the  remaining  time 
and  cost  logs. 

The  Opscan/Terminet  cluster  will  be  disconnected  by  Honeywell  and 
NCS  on  February  28  and  the  units  shipped  to  a site  designated  by 
MIISA.  Also  by  February  28th,  the  dedicated  phone  line  will  be  discon- 
nected by  Pacific  Telephone,  the  ESO  at  Stockton  will  ship  the  Course 
69  materials  and  equipment  back  to  CNTECHTRA  at  Memphis,  the  learning 
center  will  have  the  cabinets  and  LS  desk  removed,  and  the  LS  will 
leave  Stockton  to  report  for  new  duty  elsewhere.  The  LS  will  have  com- 
pleted the  CMI  student  files  and  submitted  all  the  appropriate  records 
to  CNTECHTRA  in  Memphis.  The  ESO  will  keep  whatever  is  desired  for 
Stockton  files. 

1 

For  each  Oil  student,  the  attitude  tests  and  interviews  will  be 
administered  by  PRC  staff,  or  the  ESO  and  DTC/POs,  within  one  week  fol- 
lowing completion.  On  February  28,  the  test  and  interview  will  be  ad- 
ministered by  the  PRC  staff  to  key  support  personnel. 

C.  Evaluation 

After  the  demonstration,  PRC  staff  will  analyze  the  results  in 
terms  of  the  eight  research  objectives.  In  addition,  during  the  week 
from  February  7 to  March  3 the  special  achievement  exams  will  be  taken 
at  Stockton  by  the  CMI  and  correspondence  students.  The  performance  of 
these  groups  will  be  analyzed  by  PRC  staff. 


PRC  will  have  the  following  data  and  information  in  hand  and  tabu- 
lated by  March  10: 
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• CMI  time  to  completion 

• Special  exam  scores 

e Attitude  test  scores  for  both  students  and  key  personnel 

• Attitude  interviews  for  both  students  and  key  personnel 
e Cost  data 

a Personnel  time  data  and  activity  descriptions 
a Equipment  and  Logistics  Usage 

a Interviews  with  students  and  key  personnel  on  the  operation 
of  the  CMI  course 

A total  of  four  weeks  will  be  required  to  analyze  the  data  related 
to  learning  effectiveness  and  attitudes.  The  effectiveness  will  be  ac- 
complished by  running  a standard  covariance  analysis  package  on  the  PRC 
computers  in  McLean  to  compare  CMI  student  performance  at  Stockton  to 
the  historical  Course  File  69  data  obtained  from  MIISA  Memphis.  Re- 
sults will  be  obtained  by  March  24. 

i 

I 

The  attitude  pretest  and  posttest  data  from  the  experimental  CMI 
group  at  Stockton  will  be  compared  to  the  data  from  the  CMI  control  | 

i 

group.  A covariance  analysis  will  also  be  run  on  the  PRC  computers, 

■ 

with  results  obtained  by  March  31.  The  Interview  answers  will  be  used 

to  adjust  the  Interpretation  of  the  statistical  analysis  of  the  test  I 

scores.  All  the  attitude  analyses  will  be  completed  by  April  14. 

Costs  and  personnel  time  data  will  be  used  to  assess  the  cost  ef-  ^ 

fectiveness  of  CMI  onsite  training  for  the  Navy  and  the  Department  of  j- 

Defense.  The  cost  of  major  CMI  onsite  activities  will  be  estimated  and  * 

T 

compared  to  conventional  approaches.  This  work  will  be  completed  by  <i 

June  1. 
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The  rcaalnlng  objectives  concern  personnel,  personnel  training, 
organization,  and  procedural  and  logistics  requireaents,  and  will  be 
dealt  with  in  that  order.  Each  uses  the  personnel  tine  data  and  the 
interview  information  on  problems  faced  and  actual  or  potential  solu- 
tions identified.  A case  study  approach  will  be  used  to  assess: 

• Operational  effectiveness  of  the  demonstration  design 

• Desirable  or  required  changes  to  achieve  an  effective  opera- 
tional system 

Personnel  and  personnel  training  requirements  will  be  studied  and 
conclusions  reached  by  April  7.  The  organization  and  management  struc- 
ture requirements  will  be  studied  next,  with  results  by  April  28.  Fa- 
cility and  procedure  requirements  will  be  assessed  by  May  19,  equipnent 
and  logistics  requirements  by  June  9. 

The  complete  report  on  the  demonstration  will  be  presented  in 
draft  form  July  15,  1978. 
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Appendix  A 

PRESENT  VALUE  ANALYSIS 


When  two  programs  have  different  cash  flows  over  time,  a basis  of 
comparison  is  needed  other  than  the  yearly  costs  or  savings.  Present 
value  concept  provides  such  a basis.  It  represents  the  current  value 
of  a cost  or  savings  that  will  occur  at  a specified  future  time. 

Because  it  is  possible  to  invest  money  and  obtain  income  from  it, 
receiving  $1,000  today  is  worth  more  than  receiving  $1,000  two  years 
from  now.  Specifically,  assume  that  a 10  percent  annual  rate  of  return 
can  be  obtained  for  carefully  invested  money.  $1,000  today  would  be 
worth  $1,210  ($1,000  x 1.1  x 1.1)  at  the  end  of  two  years.  Therefore, 
$1,000  is  the  present  value  of  the  $1,210  received  at  the  end  of  two 
years. 

When  deciding  between  alternative  cash  flows,  it  is  preferable  to 
choose  the  alternative  with  the  highest  present  value,  all  other  fac- 
tors being  equal.  For  example,  consider  an  offer  of  $9C0  now  or  $1,210 
in  two  years.  The  present  value  of  $1,210  is  $1,000  and  is  larger  than 
the  $900  offer.  Based  on  present  value  it  would  be  preferable  to  wait 
two  years  for  the  $1,210.  The  choice  is  confirmed  by  considering  that 
the  $900  invested  now  would  yield  only  $1,089  in  the  same  time  period. 

Costs  are  treated  in  exactly  the  same  manner.  The  present  value 
of  spending  $1,210  at  the  end  of  two  years  is  $1,000.  If  the  choice 
were  to  spend  $1,100  now  or  wait  two  years  and  spend  $1,210,  all  other 
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things  being  equal,  it  would  be  preferable  to  wait  two  years  before 
spending  the  money. 

No  matter  what  the  cash  flow  over  time,  positive  or  negative,  it 
can  be  reduced  to  a present  value  by  the  formula. 


PV  = X(0)  + 


X(l)  X(2) 


(l-^-i  {l+r)2 


where  X(n)  = cash  flow  in  year  n 

r = average  rate  of  return  for  the  year 


The  term  discount  rate  is  used  instead  of  rate  of  return  in  the 
more  general  consideration  of  public  investments  and  in  particular  in 
Defense  Department  applications.  Discount  rate  will  be  used  in  the  re- 
mainder of  this  discussion.  Often,  the  cash  flow  is  uniform  over  a 
year.  If  r is  the  rate  of  return  for  a full  year 


X(l)  X(2)  X(n) 

PV  = X(0)  + — + — + — + 

(1+f)  (1+|)2  (1+|)'' 


When  the  cash  flows  are  identical  from  year  to  year  and  are  repre- 
sented by  a 


PV  = a 


+ . . . + 


r r 2 

(i+|)  (it-|) 
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Sometimes,  when  a fixed  present  investment  is  being  compared  to  a 
cash  flow  over  time,  the  equivalent  annual  cash  flow  is  desired.  That 
is  given  by 


PV 


1 + 


<uf, 


(1+f)^ 


. + 


/I 

(1+^) 


The  selection  of  a discount  rate  r,  is  very  important  in  analyzing 
present  value.  A change  in  r could  change  the  choice  of  program.  For 
Defense  Department  programs.  Reference  3 specifies  a value  of  10 
percent. 


For  COMISAT,  a program  life  of  eight  years  (corresponding  to  the 
life  of  the  terminals)  was  selected.  For  any  component  of  the  program, 
such  as  an  individual  course  or  terminal,  the  present  value  is  calcu- 
lated as  follows 


PV  = a + 
o 


r,  2 
(I+5) 


(1^|)® 


a + 5.597(a  ) 

o c 
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Where  is  the  fixed  investment,  the  annual  cost,  the  discount 
rate  is  10  percent  and  the  program  life  is  eight  years. 

Because  the  CMI  courses  have  a life  of  12  years,  there  is  some 
value  remaining  at  the  end  of  the  eight  year  program.  Assuming  uniform 
depreciation  over  12  years,  one  third  of  the  initial  investment  remains 
at  the  end  of  eight  years.  That  value  could  be  transferred  to  another 
program  if  COMISAT  ended  and  CMI  continued  at  the  training  centers,  or 
it  could  be  charged  to  later  years  of  an  extended  COMISAT  operation 
along  with  other  annual  costs  and  savings.  Hence  the  present  value  of 
the  remaining  equity  is  an  equivalent  savings.  If  the  developing  and 
coding  cost  is 

a^  = $522,750 

as  it  is  for  a course  51  days  in  length,  then  the  remaining  equity  at 
the  end  of  eight  years  is  a^/3  = $522,750/3  = $174,250,  and  the  pres- 
ent value  of  this  equity  is 

PVq  (a../3)  = 0.4665  ($174,250)  = $81,288 
o o 


Consistent  with  the  Defense  Department  instructions  of  Reference 
3,  the  present  value  concept  was  used  to  describe  the  projected  net 
benefits  of  COMISAT.  The  reader  is  referred  to  that  document  for  fur- 
ther information  on  conducting  and  presenting  economic  analysis  results 
for  Defense  Department  programs. 
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Appendix  B 

REQUIRED  AUTODIN  II  CAPACITY 

As  discussed  in  TAEG  Report  Number  44  (page  219),  each  10  student 
hours  of  training  will  require  a transmission  of  1,087  characters  from 
the  remote  site  to  the  CMI  Computer  and  9,174  characters  from  the  CMI 
Computer  back  to  the  remote  site.  These  transmission  requirements  as- 
sume a batching  of  10  messages  together  to  increase  communication 
efficiency. 

For  purposes  of  determining  the  size  of  the  transmission  line  re- 
quired to  and  from  the  CMI  Computer,  the  larger  of  the  two  transmission 
requirements  must  be  used.  Using  the  characteristics  of  the  represen- 
tative COMISAT  program  of  Table  II-l,  the  number  of  student  training 
hours  per  year  is; 

(20,881  students/year) (.967) (255  hours)^  = 5,148,941  student  training 

hours  per  year 

Thus,  the  total  number  of  characters  per  year  to  be  transmitted  to  the 
CMI  Computer  center  is: 

(5,148,941)  (9,174)  _ 4,723,638,473  characters  per  year 
10 

= 150  characters  per  sec. 


^Assuming  an  average  course  length  of  255  hours. 
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This  transmission  requirement  should  then  be  converted  into  an  ap- 
propriate sized  transmission  link  between  the  nearest  AUTODIN  switch 
and  the  CMI  Computer  and  the  cost  of  such  a link  calculated.  Incidently 
the  net  cost  of  the  AUTODIN  lines  required  should  be  calculated  as  the 
difference  between  the  total  cost  of  the  AUTODIN  lines  and  the  cost  of 
the  additional  dedicated  lines  required  to  operate  the  CMI  skill  pro- 
gression courses  at  the  training  centers.  Neither  of  these  costs  is 
available  at  this  time. 

Similar  calculations  could  be  made  for  the  additional  transmission 
requirements  the  COMISAT  system  imposes  on  the  message  center  at  each 
remote  site.  For  the  538  sites  included  in  the  representative  case 
this  amounts  to  an  average  additional  load  of  0.28  characters  per 
second . 


r" 
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Appendix  C 

U.  S.  NAVY  TRAINING  WORKLOADS 
FY  76  through  FY  80 


footnotes 


Table  C 2.  DoD  Individual  Training  and  Education  Program  Information — Skill  Progression 


Total  Workload  695S8  68542  11563  16719  16814  10621  73638  71774  11991  76164  73532  12001 
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Appendix  D 

ESTIMATION  OF  COURSE  ENROLLMENT  DATA 


As  mentioned  in  Chapter  II,  the  data  on  individual  enrollments  for 
skill  progression  courses  was  limited  to  the  title,  length,  and  number 
of  graduates  of  the  10  courses  with  the  highest  volume.  The  lack  of 
data  necessitated  generation  of  populations  for  courses  and  approxima- 
tion of  such  enrollment  quantities  as  average  attendance  and  student 
man-days.  This  appendix  provides  further  details  about  the  procedures 
used  to  arrive  at  these  estimates. 

A.  Generating  Course  Populations 

The  basic  assumption  underlying  the  generation  of  estimates  of 
course  volumes  is  that  the  derived  populations  have  the  same  proper- 
ties, i.e.,  can  be  fitted  by  the  same  curve,  as  the  populations  of  the 
10  highest  volume  courses.  With  this  assumption,  the  required  popula- 
tions can  be  obtained  by  fitting  a curve  through  the  populations  for 
the  10  highest  volume  courses  and  extrapolating  this  curve  to  obtain 
the  needed  populations.  These  steps  are  described  in  greater  detail 
below. 

1.  Fitting  the  Curve 

As  Figure  D-1  illustrates,  the  course  populations  seem  to 
follow  an  exponential  curve.  A learning  curve  was  used  to  approximate 
these  populations  because  such  a curve  is  exponential  in  shape,  is  rel- 
atively simple  to  derive  and  manipulate,  and  provides  a close  approxi- 
mation to  the  data.  This  appendix  contains  a discussion  of  the  learn- 
ing curve  and  its  properties. 
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123456789  10 

Course  Number 


Figure  D- 1.  Popufat/on  Pfot  of  the  fO  Highest  Voiume  Courses 

A process  follows  a learning  curve  if  the  accumulated  average 
of  the  dependent  variable  decreases  to  a constant  percentage  of  the 
previous  accumulated  average  value  whenever  the  independent  variable  is 
doubled.  For  example,  if  the  populations  of  the  high  volume  courses 
follow  a learning  curve,  then,  by  doubling  the  number  of  courses,  the 
accumulated  average  student  population  is  decreased  to  a constant  per- 
cent of  the  previous  accumulated  average  population.  The  constant  per- 
centage mentioned  above  is  called  the  learning  rate.  A learning  curve 
with  a 70  percent  learning  rate  is  plotted  in  Figure  D-2. 

To  determine  if  the  course  population  data  follows  a learn- 
ing curve  the  accumulated  average  populations  for  each  course  were 

D-2 


r 


Table  D- 1.  Actual  Cumulative  and  Course  Populations 
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determined  and  used  to  examine  the  learning  rate.  Table  D-1  contains 
these  accumulated  average  populations,  which  were  found  by  dividing  the 
cumulative  population  by  the  course  number.  The  learning  rate  can  now 
he  estimated  by  performing  the  calculations  given  below  where  is 
the  accumulated  average  population  of  the  i^^  course. 


71% 

74% 

69% 

68% 

67% 


It  was  noted  that  all  of  these  ratios  are  fairly  close  to  70 
percent.  In  fact,  their  average  is  69.8  percent.  Hence,  a 70  percent 
learning  curve  seemed  to  provide  a reasonable  approximation  to  the 
course  population  data  and  was  used  in  obtaining  subsequent  course 
populations. 


The  next  step  in  arriving  at  these  course  populations  was  to 
determine  the  algebraic  form  of  the  particular  learning  curve  to  be 
used.  The  algebraic  form  of  a general  learning  curve  is 

a 


where,  for  purposes  of  this  analysis,  is  the  accumulated  average 
number  of  students  over  N courses;  a is  the  number  of  students  in 
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course  1;  N is  the  number  of  courses;  and  b is  an  exponent  associated 
with  the  learning  rate.  In  this  equation  N are  variables,  a 

is  a constant  whose  value  is  known  to  be  3310,  and  i is  a constant 
whose  value  must  be  derived. 

To  derive  b the  learning  curve  formula  was  first  applied  to 
the  second  and  fourth  courses,  which  yields 


^^2  = 


3110 

(2)^ 


and 


"4  = 


3110 


(4)' 


respectively.  Dividing  the  first  of  these  equations  by  the  second  re- 
sults in 


3110/(N) 

3110/(2N)^ 


= (2) 


1 

.70 


due  to  the  relationship  between  the  accumulated  average  populations. 
Applying  the  logarithm  function  to  both  sides  of  the  equation,  the 
value  of  b was  found  to  be  .516.  Hence,  the  specific  form  of  the 
learning  curve  which  was  applied  in  this  analysis  is 

3110 


..516 


2.  Extrapolating  the  Curve 

The  accumulated  average  population  for  the  eleventh  and  sub- 
sequent courses  can  be  determined  by  substituting  the  course  number  for 
N in  the  formula  previously  derived.  These  results  are  shown  in  Column 
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J 


f'  2 of  Table  D-2  for  courses  10  through  18.  However,  the  course  popula- 

' tion,  not  the  accumulated  average  course  population,  is  required. 

V Course  population  can  be  obtained  from  the  accumulated  average  popula- 

tion by  employing  the  following  steps: 

o For  course  10  and  all  subsequent  courses,  multiply  the  ac- 
cumulated average  by  the  course  number,  N,  to  find  the  cumu- 
lative number  of  students  in  each  of  the  N courses.  The 
\ results  of  these  calculations  are  shown  in  Column  3 of  Table 

D-2. 

o Subtract  the  cumulative  population  for  the  N-1®^  course 
■ from  the  cumulative  population  for  the  N^^  course  to  find 

the  number  of  students  taking  the  n'"^  course.  The  result- 
ing course  enrollments  can  be  found  in  Column  4 of  Table  D-2. 


i.  Table  D-2.  Approximate  Cumulative  and  Course  Populations 


(1) 

(21 

(3) 

(4) 

N 

Course  Number 

Accumulate 

Average 

Fitted 

Cumulative 

Course 

Population 

10 

947,9 

9,479 

11 

902.4 

9,926 

447 

12 

862.8 

10,353 

427 

13 

827.9 

10,762 

409 

14 

796.8 

11,155 

393 

15 

768.9 

11,534 

379 

16 

743.8 

11,900 

366 

17 

720.9 

12,255 

355 

18 

699.9 

12,598 

343 

B.  Approximation  of  Enrollment  Quantities 

Two  enrollment  quantities  were  approximated 
average  attendance  and  student  man  days. 


in  this  analysis: 
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1 . Average  Attendance 

The  average  number  of  people  enrolled  in  a course  at  any 
given  time  was  determined  from  the  number  of  graduates,  using  data  from 
Table  II-5.  The  average  number  of  students  is: 


entrants  graduates  _ 62,909  + 61,666 
2 2 


62,287 


Average  attendance  can  be  obtained  from  the  number  of  graduates  by: 


total  average  attendance 
total  number  of  graduates 


(number  of  graduates  in  the  course) 


= 62,287  (course  graduates)  = 1,01  (course  graduates) 

61,666 


2.  Student  Man  Days 

The  number  of  student  man-days  required  by  a course  was  de- 
termined by  multiplying  the  average  attendance  by  the  course  length. 
With  the  exception  of  courses  one  through  ten,  all  courses  were  assumed 
to  require  51  days,  the  average  course  length  for  skill  progression 
courses . 


The  results  obtained  by  these  procedures  were  used  in  the 
economic  analysis  in  Chapter  II. 
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Appendix  E 

DETAILS  OF  THE  CALCULATION  OF  COSTS,  SAVINGS, 
BREAK-EVEN  POPULATIONS 


The  components  of  costs  and  savings  were  described  in  general 
terms  in  Chapter  II.  This  appendix  presents  the  details  of  the  tech- 
niques used  to  calculate  each  cost  element  so  that  the  reader  may  ver- 
ify the  analytical  results. 

A.  Costs  and  Savings  Incurred  Through  Development  of  CMI  Courses  for 

Training  Center  Use 

1 . Course  Development,  Coding  and  Maintenance 

As  discussed  in  Chapter  II,  developing  a course  for  CMI  in- 
struction and  coding  it  for  use  on  the  computer  requires  an  incremental 
expenditu'-e  of  $2,050  per  hour  of  training.  The  number  of  training 
hours  in  a course  was  determined  by  first  multiplying  the  number  of 
calendar  days  by  5/7,  to  obtain  the  number  of  working  days  required  by 
the  course.  Multiplying  the  result  by  7,  the  number  of  training  hours 
in  a day,  yielded  the  hours  of  training  required  by  a course.  The  de- 
velopment and  coding  cost  were  then  found  by: 

Development  and  coding  cost  per  course: 

= ($2,050)  (calendar  days)  (5/7)  (7)  111 

= ($10,250)  (calendar  days) 

The  annual  course  maintenance  cost  was  estimated  by  taking  five  percent 
of  this  incremental  expense  of  development  and  coding. 
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2 . Computer  Expansion  Cost 

, The  components  of  computer  expansion  costs  were  also  discus- 

» sed  in  Chapter  II  and  an  annual  cost  of  $14,400  for  each  thousand  AOB 

students  was  determined.  AOB  can  be  found  by  dividing  the  average  an- 
; nual  attendance  by  the  number  of  successive  courses  in  a year,  which  is 

365  divided  by  the  course  length  in  calendar  days.  Therefore,  AOB  is: 

. (average  annual  attendance)  ^course  length  in  daysj 

365 

and  annual  computer  leasing  cost  is: 

(— ^-^  ^ ) (average  annual  attendance)  length  in  days^ 

■ 1,000  365 

= $.0395  (average  annual  attendance) (course  length  in  days)  12) 

3 . Terminal  Costs 

For  each  60  AOB  students  taking  CMI  at  the  training  center  an 
additional  terminal  must  be  purchased  and  maintained.  Assuming  courses 
can  share  terminals,  the  number  of  training  center  terminals  required 
to  service  a course  can  be  found  by  dividing  the  number  of  AOB  students 
in  the  course  by  60. 

The  investment  and  maintenance  costs  of  a training  center 
terminal,  from  Table  II-7  in  Chapter  II,  are  $14,250  and  $147/month  re- 
spectively. The  eight  year  present  value  of  total  cost  of  one  training 
center  terminal  is: 

$14,250  + 5.597  ($147)  12  = $24,123. 

Therefore,  the  present  value  of  a training  center  terminal  cost  for  any 
course  can  be  found  by  : 
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1 V 


Course  AQB  ($24,123)  131 

60 

4 . Savings  in  Student  Time  and  Living  Costs 

The  saving  in  student  man-hours  of  training  with  CMI  compared 
to  conventional  training  is  the  product  of  the  number  of  AOB  students, 
the  percentage  of  time  saved  by  CMI,  and  the  annual  rate  of  pay  for 
each  student.  The  annual  salary,  including  basic  pay,  quarters,  incen- 
tive and  special  pay,  retirement  pay,  and  miscellaneous  expenses,  was 
assumed  to  be  $10,090,  the  salary  of  a student  with  an  E-4  rating.  As 
indicated  in  Chapter  II,  a 20  percent  decrease  in  training  time  was  as- 
sumed. Thus,  the  annual  saving  in  student  salary  is: 

(AOB)  (.20)  ($10,090)  = $2,018  (AOB)  (4) 

Incremental  living  savings  was  derived  in  a similar  manner  by 
multiplying  the  average  annual  number  of  students,  average  number  of 
days  saved,  and  incremental  living  expense  saved.  The  average  number 
of  days  saved  was  calculated  as  the  product  of  the  number  of  calendar 
days  in  a course  and  the  20  percent  time  savings  factor.  The  incremen- 
tal living  cost  saved  is  $2.50  per  day.  So  the  annual  living  cost  sav- 
ings was  determined  by: 

(average  number  of  students)  (.20)  (calendar  days)  ($2.50) 

= $.50  (average  number  of  students) (calendar  days)  151 

B . COMISAT  Costs  and  Savings  for  Training  Center  Justified  Courses 

1 . Terminal  Cost 

In  order  to  teach  the  first  17  courses  through  COMISAT,  the 
only  costs  are  for  terminal  purchase  and  maintenance.  For  the  low 
maintenance  case,  the  undiscounted  terminal  investment  cost  is  the  num- 
ber of  sites  multiplied  by  the  unit  terminal  price  of  $11,450.  For  the 
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2.  Maintenance  Costs 

Maintenance  costs  depend  on  the  site  category  and  the  main- 
tenance assumptions  of  each  case.  However,  with  the  exception  of  Cate- 
gory ],  (CONUS  sites  50  miles  or  less  from  a service  location)  unit 
maintenance  rates  for  the  high  maintenance  case  can  be  obtained  from 
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those  of  the  low  maintenance  case  by  multiplying  by  4/3  for  land  sites 
and  6/4  for  ship  sites.  Hence,  this  discussion  will  be  concerned  only 
with  the  low  maintenance  case. 

Category  1 sites  would  be  serviced  by  Opscan  at  a cost  of  $80 
per  month  or  $960  per  year.  Category  2 sites  (CONUS  sites  between  50 
and  90  miles  from  service)  would  also  be  maintained  by  Opscan  but  on  a 
time-and-mater ials  basis  which  involves  costs  of  travel,  labor  and 
parts.  Travel  charges  are  18  cents  per  mile  plus  $30  per  hour  of  tra- 
vel for  each  failure.  If  an  average  travel  speed  of  45  mph  is  assumed, 
the  average  total  travel  cost  per  year  would  be: 

(number  of  annual  breakdowns)  ($.18)  (miles  to  the  site)  + (number 
of  annual  breakdowns)  (1/45)  ($30)  (miles  to  the  site) . 

Since  this  cost  will  vary  with  the  distance  to  the  site,  a mean  travel 
cost  of  $2,218  for  all  sites  in  this  category  was  calculated.  Labor 
cost  is  $30  per  hour  and  the  mean  time  to  repair  is  2 1/2  hours;  thus 
the  average  annual  labor  cost  is: 

($30)  (2  1/2)  (number  of  annual  breakdowns)  = $75  (number  of 

annual 

breakdowns) 

The  only  information  available  concerning  the  cost  of  parts 
was  Opscan 's  price  of  $600  for  a service  warranty  for  the  second  year. 
This  was  taken  as  the  annual  parts  cost.  Naval  personnel  would  service 
all  other  remote  (Category  3,  4,  and  5)  sites,  thereby  incurring  costs 
of  labor  and  parts.  The  total  labor  cost  per  repair  for  service  by 
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site  personnel  was  assumed  to  be  the  same  as  Opscan's  total  labor 
cost.^  Therefore,  labor  cost  is,  as  before: 

$75  (number  of  annual  breakdowns) . 

The  average  annual  cost  of  replacing  used  parts  from  the  inventory, 
based  on  a failure  rate  of  three  per  year,  was  estimated  by  assuming 
that  Opscan's  $600  per  year  warranty  price  was  derived  on  the  basis  of 
the  price  for  parts  used  and  their  expected  return  on  investment  for  a 
perpetual  inventory.  It  was  further  assumed  that  they  require  a 30 
percent  return  on  their  parts  investment  of  $6,529  and  that  this  inven- 
tory is  used  by  the  Opscan  repairmen  in  servicing  10  sites.  The  de- 
sired return  on  investment  is  $1,960  or  approximately  $200  per  year  per 
site.  Thus,  the  average  cost  of  parts  to  the  Opscan  company  is  $400 
per  year  for  each  terminal  when  three  annual  failures  occur.  The  same 
parts  cost  was  assumed  for  service  by  Navy  personnel. 

3.  COMISAT  Savings 

The  savings  achievable  by  utilizing  COMISAT  to  teach  the 
first  17  courses  consist  of  three  components:  student  travel,  living 
expense  at  the  training  center,  and  the  cost  of  terminals  no  longer  re- 
quired at  the  training  center.  The  average  cost  of  a one-way  trip  to  a 
training  center  is  $137.  In  addition,  a student  on  PCS  is  allowed  $291 
to  cover  transfer  of  household  effects.  Thus,  the  total  travel  costs 
saved  per  student  for  PCS  and  for  TDY  are,  respectively: 

$141.50  + $291  = $432.50  and 

2($141.50)  = $283. 


This  is  a conservative  assumption,  permitting  the  ET  to  take  a longer  ] 

mean  time  to  repair  than  the  Opscan  repairman  since  the  ET's  hourly  j 

cost  is  less.  j 

1 

J 
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The  average  travel  cost,  taking  into  account  the  relative  frequency  of 
PCS  and  TDY^  is; 

($432.50)  + (liilil)  ($283)  = $354. 39/entrant , or 
62,909  62,909 

($354.39)  = 1.02($354.39)  = $361 . 48/graduate . 

61,666 

The  total  travel  savings  is; 

($361.48)  (number  of  graduates).  |6| 

The  living  savings  is  the  product  of  the  number  of  student- 
days  and  $2.50,  the  daily  living  allowance  at  the  training  center. 

There  is  a savings  of  one  training  center  terminal  for  every  60  AOB 
students  reached  through  COMISAT.  Each  training  center  terminal  re- 
quires a $14,250  investment  and  a present  value  cost  of  $9,873  to  main- 
tain over  eight  years.  Hence  the  savings  in  training  center  terminals 
is; 

^ ($14,250  + $9,873)  = $402.05  AOB.  |7| 

60 

C.  Calculation  of  Break-Even  Populations 

In  order  to  determine  which  sites  should  be  outfitted  with  COMISAT 
terminals  it  was  necessary  to  calculate  the  population  for  which  the 
cost  of  delivering  training  to  the  site  is  equal  to  the  savings 
achieved,  because  all  site  populations  greater  than  this  would  result 
in  savings.  This  section  describes  the  derivation  of  the  break-even 
population  for  each  cagegory  of  site. 


1 


The  data  in  the  proportions  come  from  Table  II-5. 
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The  formula  used  to  derive  the  break-even  population  is: 

= TC  E/TGS 
be  pv  ' pv 

where  = break-even  population 

= present  value  of  equipment 
E = enlisted  personnel  population 

= present  value  of  travel  and  living  savings 

The  present  value  of  total  gross  travel  and  living  savings  for 
those  courses  economically  justified  for  use  at  the  training  centers 
was  found  to  be  $23,714,343  when  discounteo  at  10  percent  over  eight 
years.  The  total  enlisted  personnel  population  (E)  was  given  as 
370,346.  Thus,  the  only  quantity  needed  to  determine  the  break-even 
population  (P^^)  was  the  present  value  of  the  total  cost  of  a site 
discounted  over  eight  years,  '^pv  termi- 

nal and  spare  parts  cost  without  quantity  discounts,  and  the  eight  year 
present  value  of  maintenance.  Two  factors,  total  number  of  terminals 
purchased  and  site  category,  influence  the  value  of  TCpy.  For  this 
reason,  a different  break-even  population  was  calculated  for  each  cate- 
gory within  each  case. 

The  only  exception  to  the  calculation  of  break-even  population  by 
site  category  occurs  in  Category  2,  where  the  dependence  of  maintenance 
cost  of  mileage  makes  calculation  of  one  for  the  entire  category 

impossible.  As  a result,  decisions  about  the  inclusion  of  Type  2 sites 
were  made  by  comparing  the  total  cost  of  each  particular  site  to  the 
savinos  which  it  could  contribute. 

Since  quantity  discounts  are  available  for  terminals  and  spare 
parts,  the  investment  cost,  and  hence  the  total  cost,  depends  on  the 
numbers  of  terminal  and  parts  sets  required.  For  this  reason,  the 
calculations  were  made  iteratively  using  the  following  steps: 
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• Estimate  the  discount  intervals  in  which  the  number  of  ter- 
minals and  the  number  of  parts  sets  will  fall. 

• Using  these  estimates  determine  the  correct  quantity  discount 
factor  from  Table  II-IO. 


• Use  the  discount  factors  to  derive  a quantity  discounted  in- 
vestment cost  of  a terminal  and  a spare  parts  set. 


Determine  TC 
tenance  cost^? 


by  adding  the  investment  costs  to  the  main- 
or  the  category  under  consideration 


Use  the  equation  given  above  to  derive  P 


be* 


Compare  the  site  populations  to  to  identify  sites  whose 
population  is  larger  than  P These  are  the  sites  fea- 
sible for  COMISAT  development.  If  the  number  of  terminals 
and  spare  parts  are  within  the  estimated  intervals,  the  cor- 
rect break-even  populations  have  been  calculated.  Otherwise, 
revise  the  interval  estimates  as  indicated  and  repeat  the 
process . 


This  procedure  resulted  in  the  identification  of  306  feasible 
sites  requiring  an  expenditure  of  $5,348,000  in  present  value  over 
eight  years. 

D.  Costs  and  Savings  in  the  Iterative  Approach 
1 . Costs 

For  courses  whose  development  is  justified  only  by  COMISAT, 
the  cost  of  development,  coding,  maintenance,  leasing,  site  terminals, 
and  the  cost  of  additional  training  center  terminals  required  by  stu- 
dents who  can't  be  reached  by  COMISAT  must  all  be  charged  to  COMISAT. 
With  the  exception  of  training  center  costs,  each  of  these  is  calculat- 
ed as  described  in  Sections  A and  B of  this  appendix.  Terminal  costs 
for  implementing  70  percent  of  the  18th  and  subsequent  courses  at  the 
training  center  are  derived  similar  to  the  savings  discussed  in  Section 
B.  However,  the  number  of  students  who  must  take  the  new  courses  at 
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the  training  center  (AOB^)  is  used  in  the  formula.  Thus,  the  train- 
ing center  terminal  cost  charged  to  COMISAT  is: 

) ($14,250  + $9,873) 

60 

where  AOB^  is  the  number  of  students  who  cannot  be  reached  at  their 
own  sites  by  the  18st  and  subsequent  courses. 

The  gross  COMISAT  cost  for  teaching  all  economically  and  op- 
erationally feasible  courses  at  Navy  sites  is  summarized  in  Table  II-4 
and  can  be  found  by  evaluating  the  following  expression: 

Eight  year  present  value  of  COMISAT  gross  cost  = 

(development  and  coding  for  courses  justified  only  by  COMISAT  111)+ 
(computer  leasing  121 ) + (course  maintenance  for  courses  justified 
only  by  COMISAT)  + (COMISAT  terminals)  + (training  center 
terminals  131 ) . 

Applying  the  appropriate  formula  for  each  cost  yields: 

Eight  year  present  value  of  COMISAT  gross  cost  = 

.7(N-17)  (2,050)  (51)  (5/7)  (7)  - . 7 (N-17)  ($81 , 288)  + 

.7(M-17)  (5.597)  (.05)  (522,750)  + (^y'-^"°-)  (g^^J  (1.01)  (^)  + 

COMISAT  terminal  cost  + ($14,250  + $9,873), 

60 
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where  N is  the  total  number  of  economically  feasible  courses  of  which 

70  percent  can  be  converted  to  CMI.  a . 

y number  of  graduates 

in  the  70  percent  of  Courses  18  through  N convertible  to  CMI,  and  PC  is 
the  percent  of  the  population  which  cannot  be  reached  onsite  through 
COMISAT.  The  above  expression  can  be  simplified  to: 

Eight  year  present  value  of  COMISAT  gross  costs  * 

411,426N  + 11.4  g + 57  (g  (PC)  + COMISAT  terminal  cost 


2 . Savings 

The  savings  COMISAT  achieves  through  development  of  addition- 
al courses  consist  of  travel,  living,  time,  and  incremental  living  sav- 
ings. Each  of  these  is  calculated  using  formulas  discussed  previously 
in  this  appendix. 

I 

The  gross  COMISAT  savings  for  teaching  70  percent  of  the  N 
highest  volume  courses  at  remote  sites  is: 

Eight  year  present  value  of  gross  COMISAT  savings  » (travel 
cost  for  students  of  all  courses  as  indicated  in  equation  (61 ) 

+ (living  costs  for  students  of  all  courses)  + (time  costs 
for  students  of  courses  18  to  N as  indicated  in  equation  14]) 

+ (living  costs  for  students  of  courses  18  to  N as  indicated 
in  equation  15])  + (training  center  terminal  costs  for  stu- 
dents who  are  taught  Courses  1 to  17  through  COT^ISAT  as  indi- 
cated in  equation  (31  ) . 

Substituting  the  proper  formulas  into  this  expression  yields: 

Eight  year  present  value  of  gross  COMISAT  savings  * 

(5.597)  (1-PC)  ($361.48)  (g)  + 5.597  (1-PC)  ($2.50)  (51)  (g)  (.80)  t- 

(5.597)  9 01)  (51/365)  (.20)  ($10,090)  + 

9 (-20) (51) ($2.50) (5.597)  + 

training  center  terminal  costs 
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where  g is  the  total  number  of  graduates  in  the  N courses  convertible 
to  CMi . 


This  expression  simplifies  to: 

Eight  year  present  value  of  gross  COMISAT  savings  * 2,594  g - 
2,594  g (PC)  + 1,741  9 + training  center  terminal  cost. 

Subtracting  the  gross  savings  expression  from  the  gross  cost 
expression  results  in  net  cost.  A simplified  expression  for  net  cost 
in  terms  of  N can  be  obtained  by  using  the  identities: 

,iN  = -7  (3,110  110(17)  -484) 

» 2,177n.484-8,570 


But,  by  definition. 


g 


* 9,334  + g 

iN 

- 2,177n.484  + 755 


The  simplified  net  cost  expression  is: 

Eight  year  present  value  of  net  COMISAT  cost  = 411,426n  - 
6,107,356N.484  4 2,342,017N-484pc  + 850,425PC  + 7,497,450 
+ COMISAT  terminal  cost  + training  center  terminal  cost. 

PC,  COMISAT  terminal  cost,  and  training  center  terminal  cost  are  )cnown 
once  the  number  of  sites  is  fixed.  Thus,  net  cost  is  a function  of  the 
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n 


The  expression  for  net  cost  is  useful  in  two  ways.  First,  by 
taking  its  derivative,  setting  the  result  to  zero,  and  solving  for  N, 
the  maximum  number  of  courses  which  could  be  justified  for  CMI  is  ob- 
tained. The  net  cost  expression  can  also  be  rewritten  in  terms  of  the 
student  time  saving  and  operational  constraint  parameters.  With  the 
expression  in  this  form  the  sensitivity  of  total  cost  to  the  parameters 
can  he  easily  investigated.  The  results  of  this  sensitivity  analysis 
were  shown  for  the  low  maintenance  case  in  Table  II-3.  Similar  results 
for  the  high  maintenance  case  can  be  found  in  Table  E-2. 


Table  E-2.  Sensitivity  of  Present  Value  Savings  to  Time  Savings  and 
CM!  Convertibility  Assumptions:  High  Maintenance  Case 


Percentage  of  Skill  Progression  Courses 
Convertible  to  CMI 

Percentage  of  Student  Time  Saved  | 

10% 

15% 

20% 

60% 

$14,466,000 

$18,079,000 

$21,691,000  1 

1 

70% 

$18,423,000 

$22,637,000 

$26,852,000 

00 

o 

$22,379,000 

$27,196,000 

$32,012,000 
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Appendix  F 

DETERMINATION  OF  MAXIMUM  MILEAGE 
FOR  MAINTENANCE  BY  OPSCAN  PERSONNEL 

In  order  to  determine  the  total  maintenance  costs,  a decision  must 
be  made  regarding  which  CONUS  sites  should  be  serviced  by  Opscan  per- 
sonnel and  which  by  Naval  (ET)  personnel.  The  cost  of  service  by  an  ET 
is  constant  within  each  of  the  maintenance  cases  defined  in  the  report, 
and  the  cost  of  Opscan  service  varies  only  with  the  mileage  to  the  site 
from  the  service  location.  Hence,  all  sites  beyond  a determined  dis- 
tance from  service  should  have  site  personnel  handle  repairs;  all  other 
CONUS  sites  should  be  serviced  by  Opscan. 

The  break-even  mileage  for  each  case  can  be  determined  by  perform- 
ing the  following  steps.  The  low  maintenance  case  data  are  used  as  an 
illustrative  example. 

• Sum  the  costs  of  parts,  labor,  and  inventory  investment  to 
arrive  at  an  eight  year  present  value  cost  of  ET  repair. 

Cost  of  parts  for  ET  repair  = S400/year 

Cost  of  labor  for  ET  repair  = 3 (2.5) $30  = $ 225/year 

Inventory  Investment  = $6,529 

Eight  year  present  value  cost  of  ET  repair  = $6,529  + 5.597  ($625) 

= $10,027 

• Sum  the  costs  of  parts,  labor,  and  mileage  to  arrive  at  an 
expression  for  the  eight  year  present  value  cost  of  Opscan 
service  as  a function  of  miles  to  the  site. 

Cost  of  parts  for  Opscan  service  = $600/year 

F-1 


TAEG  Report  No.  49 


1 


VI 


Cost  of  labor  for  Opscan  service  = 3(2.5)$30  = $225/year 

Cost  of  mileage  for  Opscan  service  * ($.18)(6)d  + 

(1/45) (6)  (S30)d 

= $1.08d  + $4d 

= $5.08d 

where  d is  the  mileage  from  the  service  location  to  the  site. 

Eight  year  present  value  cost  of  Opscan  service 

= 5.597($825  + $5.08d) 

= $4,617  + $28. 43d 

• Equate  these  two  costs. 

$10,027  = $4,617  + $28. 43d 

• Solve  the  resulting  equation  from  the  desired  mileage, 

$10,027  - $4,617 
" $28.43 

d = 190  miles 

Similar  calculations  for  the  high  maintenance  case  result  in 
equally  high  mileages.  However,  response  time  for  this  distance  would 
be  several  hours.  To  reduce  the  terminal  down  time  to  more  reasonable 
limits,  a maximum  distance  of  90  miles,  with  a two  hour  driving  time, 
was  chosen. 
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Appendix  G 

AIRLINE  SCHEDUI£  OF  TARIFFS^ 

The  following  is  the  current  procedure  used  by  all  airlines  to 
calculate  the  gross  fare  including  tax  as  a function  of  flight  distance. 

1.  Insert  a fixed  Base  Terminal  charge  of  $16.05. 

2.  Add  to  this  a series  of  cumulative  charges  relating  to  the 
flight  distance  involved^,  as  follows: 

0 - 500  miles:  $ .0877/mi. 

501  - 1500  : $ .0669/mi. 

1501  + ; $ .0643/mi. 

3.  Next  include  the  8%  tax  by  multiplying  the  calculated  rate  by 
1.08  and  round  up  or  down  to  the  nearest  dollar. 

4.  From  this  total  fare,  the  airline  receives  the  total  fare  di- 
vided by  1.08.  The  U.S.  Government  receives  the  remaining, 
e.g.,  calculate  the  fare  on  a flight  of  1,500  miles: 


^Reference  Civil  Aeronautics  Board,  Bureau  of  Tariff 
^If  a carrier  flies  nonstop,  the  point  distance  is  used.  If  the 
flight  has  intermediate  stops,  the  mileage  between  the  two  points  con- 
sists of  the  sum  of  the  most  direct  logs  available.  However,  if  an- 
other airline  has  a direct  flight  at  a lower  cost,  the  competing 
airline  will  reduce  his  rate  to  that  of  the  nonstop  airline  to  compete. 
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1. 

$^^.05 

2. 

$43.85 

$66.90 

Total 

$126.80 

3. 

(1.08)  ($126. 

80)  = $136.94  which 

is  rounded  up  to  $137. 

4. 

From  this  the 

airline  receives: 

$137/1.08  = $126.85  and 

the  remainder 

($10.15  is  the  tax) 
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Appendix  H 

RESEARCH  DESIGN  DATA  FORMS 


The  appendix  contains: 


The  Student  Registration  Card 


• The  Attitude  Survey  Instruction  Script 


The  Data  Tally  Sheet 


• Data  Input  Format:  Attitudes 
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Figure  H 1.  Student  Registration  Card 
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V. 

: ATTITUDE  SURVEY  INSTRUCTIONS 

You  are  about  to  complete  a survey  designed  to  help  the  Navy  get 
information  about  your  feelings  toward  the  training  you  are  currently 
receiving  at  this  command.  Before  you  complete  the  survey,  we  would 
like  to  ask  you  to  take  a few  minutes  to  read  the  information  and  di- 

Irections  on  the  first  page  of  the  booklet. 

Remember,  this  is  not  a test;  there  are  no  right  or  wrong 

1 

I answers.  Your  survey  will  be  scored  by  personnel  outside  of  this  com- 

j mand  and  only  statistical  group  results  will  be  shown  to  anyone  in  your 

t command.  We  are  interested  in  your  honest  feelings  about  the  training 

i 

program.  After  reading  the  directions  for  this  survey,  if  you  have  any 
questions,  please  don't  hesitate  to  ask  before  you  start  the  survey. 
Take  your  time  but  try  to  answer  each  item  with  your  immediate  feel- 
ings. Answer  each  item  as  honestly  as  you  can.  Remember,  there  are  no 
right  or  wrong  answers,  we  just  want  to  know  how  you  feel. 
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Figure  H 3.  Data  Input  Format.  Course  Performance 

where  x « the  cover  ate  entries 
^*2^  » BTB  score 
’*2  “ reading  ability  scores 
y = the  dependent  measure  entries 

^2  “ BE/E  course  performance 

^2  “ special  electricity  and  electronics  exam  score 

k * group  identification  (CMI,  correspondence,  no  formal 
training) 


H-5 


TAEG  Report  No.  49 


xxxxxxxxxxxxx  yyyyyyyyyyyyy 


1 1 

1 0 0 0 M M 0 0 D S 0 n 0 Q 0 0 0 0 D 

0 0 Q G 

1 

c 0 Q : c 

1 

0 0 0 Q G Q G 0 0 a 0 C 0 0 

0 G 0 0 0 1 0 

c 0 C C 0 0 0 0 C 0 

G 0 0 C 0 D 

0 G S 

0 0 Q Q 

C 0 0 0 

■ > 1 t » t ■ 1 iifli'UDMnitiivinrn 

1 11  1 M 1 M M 1 1 1 M M 7 1 ) 1 

;w*  r » 

MM 

V 71  ;«  K » 

Mill 

t;  t)  « M ' n H »«  « 47  «;  *.  • 

I 1 n 1 M 1 M I 1 M 

4t  < 4<  41  ^ « W 

1 1 1 1 1 11 

U>4U»-  MMKii: 

1 I 1 1 1 i 1 1 1 1 

9:  M U U k'  w 

11)111 

0 re  n 

1 1 1 

n n >•  1 

Mil 

t ’ T 1 

Mil 

7 ? 7 ? ? ? 7 2 ? 7 ? ? ? 7 ? 7 ? 7 ? ? 2 7 

7 7 7 7 

7 7 7 7 7 

77777277777777 

7 7 7 7 7 7 7 

7 7 ? 7 7 7 7 7 7 7 

7 : 7 7 7 7 

7 7 7 

7 7 7 7 

7 7 7 7 

3 ] 3 1 3 3 3 3 ] ] n 3 3 3 3 3 ] ] 3 3 3 

3 3 3 3 

3 3 3 3 3 

3333333333333: 

: 3 3 3 3 3 3 

3333333333 

3 3 3 3 3 . 

3 3 3 

3 3 3 3 

3 3 3 3 

« 4 4 ( M 4 4 4 4 ( 4 4 4 4 4 4 4 4 4 4 4 

4 4 4 4 

4 4 4 4 4 

4 4 4 4 4 4 4 4 4 4 4 4 4 4 

4 4 4 4 4 4 4 

4 4 4 4 4 4 4 4 4 4 

4 4 4 4 4 4 

4 4 4 

4 4 4 4 

4 4 4 4 

b i S S 5 S 5 E ! b i i 5 : i i 5 b b b b 

b b b 2 

b 5 b b b 

b b b b b b b b b b b b b b 

b i b b b b b 

b b b b b b b b b b 

b b b b b b 

b b b 

b b b b 

b b i b 

7 n M ’ M ? ? ? ? ? M M I M ? ? ’ • ’ / : j M M I n n j n : ? ] 1 1 7 M ’ I M 1 ? ] 7 M n J ' M I I j ? j M M • I ' M ] I 


1 n M ! 

! S G M E f E 

: 8 E 9 3 E ? S ! 9 S 

E t : B E s 

9 8 1 ! E c 

E E E 3 £ c ? : i 

! L : 8 E « 8 3 E t c ' 

' c E 6 . : 

* C 9 ' f 

M 8 ! 1 

9 9 8 9 9 : 

: : 9 5 S E 1 9 

: 9 ? 9 9 9 5 9 . : '1 

C t 3 

3 “ b 9 3 

t 0 Q 0 0 ; : 3 1 

5 ■ : ' 5 4 M * : 

f 3 9 b 

] ? 9 9 ' 

. 3 9 9 « 

4 j 1 

' 

Figure  H-4.  Data  Input  Format:  Attitudes 


where  x = covariates  (pretest  attitude  scores  for  each  attitude 
component) 

y = dependent  measures  (posttest  attitude  scores  for  each 
attitude  component) 
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Appendix  I 

INTERVIEW  INSTRUMENTS 


This  appendix  contains: 

• The  Semantic  Differential  Survey  Instrument  for  Students 

• The  Semantic  Differential  Survey  Instrument  for  Key  Support  Personnel 

• The  Post  Interview  Questions  for  Students  and  Key  Personnel 
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Appendix  I,  Exhibit  1 
THE  SEMANTIC  DIFFERENTIAL  SURVEY 
INSTRUMENT  FOR  STUDENTS 


DIRECTIONS:  On  the  following  pages  we  are  asking  you  to  rate  your  feelings 
toward  a number  of  concepts/things  related  to  your  training.  In  accordance 
with  SECNAVINST  5211.5  of  14  Aug  1975,  information  included  in  the  CMI  survey 
is  requested  under  the  authority  of  Departmental  Regula- 

tions. This  information  is  required  to  assist  in  the  evaluation  of  the  CMI 
System.  You  are  not  required  to  provide  this  information,  but  you  can  be  as- 
sured that  your  individual  answers  will  be  kept  completely  confidential  and  no 
attempt  will  be  made  to  evaluate  you  or  the  command  you  are  currently  with 
based  on  your  answers. 

On  the  following  pages  you  are  asked  to  rate  your  feelings  on  the  following 
concepts:  COURSE  MATERIAL,  COMPUTER  MANAGED  INSTRUCTION,  LEARNING  SUPERVISOR, 
STUDY  CENTER,  CMI  TESTING  PROCEDURES,  CONTENT  OF  CMI  TEST  FEEDBACK,  CMI  TEST 
FEEDBACK  TIME. 

Here  is  how  to  use  the  scales: 


If  you  feel  the  scale  relates  very  closely  to  your  feelings  mark  it  as  such: 


strong  ^ 


strong 


weak 


or 


X weak 


If  you  feel  the  scale  relates  quite  closely  to  your  feelings  mark  it  as  such: 


strong 

strong 


weak 


weak 
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If  you  feel  the  scale  relates  only  slightly  to  your  feelings  mark  it  as  such: 

strong  : : X : : : : weak 

or 

strong  : : : : X : : weak 


If  you  feel  you  are  neutral  or  feel  the  scale  is  not  relevant  to  your  feelings, 
mark  it  as  such: 

strong  : : : X : : : weak 


PLEASE  remember  THE  FOLLOWING: 


1.  Mark  all  scales;  do  not  skip  any. 

2.  Place  your  mark  in  the  middle  of  each  line,  not  in  between  lines. 

3.  Work  quickly  and  do  not  put  more  than  one  mark  for  each  scale. 

4.  Please  be  sure  that  you  have  filled  in  the  necessary  information  on  the 
cover  page  of  the  survey. 
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THE  LEARNING  SUPERVISOR 


informed 

uninformed 

bad 

qood 

available 

unavailabl 

valuable 

worthless 

effective 

ineffecti v 

inexpert 

expert 

helpful 

obstructiv 

unqualified 

qualified 

unreliable 

reliable 

unorqanized 

, 

« 

• 

organized 

Coiranents:  Please  comment  on  what  it  is  that  you  like  best  about  the  concept 
you  have  rated  above  and  what  it  is  you  like  least. 

Like  best: 


Like  least: 


Other  comments  (use  other  side  if  necessary) ; 
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THE  COURSE  MATERIAL 


valuable 


inadequate 
meaningful 
bor i ng 


frustrating 


unorganized 


approve 


useful 


dislike 


worthless 


adequate 

meaningless 


interesting 


motivating 


organized 

disapprove 


useless 


Comments:  Please  comment  on  what  it  is  that  you  like  best  about  the 
concept  you  have  rated  above  and  what  it  is  you  like  least. 


Like  best; 


Like  least: 


Other  comments  (use  other  side  if  necessary) : 
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V 


THE  STUDY  CENTER 


cramped 

; 

roomy 

satisfactory  

unsatisfact 

uncomfortable 

, 

comfortable 

dislike  

like 

organi zed 

unorganized 

pleasant  

unpleasant 

noisy  

quiet 

disapprove  

approve 

adequate  

inadequate 

awful 

nice 

Comrents:  Please  cominent  on  what  it  is  that  you  like  best  about  the 

concept  you  have  rated  above  and  what  it  is  you  like  least. 

Like  best; 


Like  least: 


Other  comments  (use  other  side  if  necessary): 


1-6 


TAEG  Report  No.  49 


COMPUTER  MANAGED  INSTRUCTION  CMI 


like 

: 

dislike 

valuable 

worthless 

adequate 

inadequate 

meaninqless 

, 

meaningful 

interesting 

boring 

frustrating 

motivating 

good 

bad 

unorganized 

organized 

disapprove 

approve 

useless 

useful 

Comments:  Please 

comment  on 

what  it  is 

that 

you 

like  best 

about  the 

concept 

you 

have 

rated  above 

and 

what 

it  is  you 

like  least. 

Like  best: 


Like  least: 


Other  comments  (use  other  side  if  necessary) : 
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V 

/ 

\ 


I 

i 

i 

i 

( 


1 


inefficient 

: 

: 

efficient 

disapprove 

approve 

right 

wrong 

effective 

ineffective 

organized 

* 

unorganized 

dislike 

like 

vaque 

• 

clear 

fair 

unfair 

bad 

• 

• 

good 

adequate 

• 

• 

inadequate 

Conunents : 


Like  best: 


Please  connnent  on  what  it  is  that  you  like  best 
concept  you  have  rated  above  and  what  it  is  you 


about  the 
like  least. 


I 

I 


Like  least: 


Other  conunents  (use  other  side  if  necessary) : 
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CONTENT  OF  CMI  TEST  FEEDBACK 


good 

• 

• 

bad 

dislike 

• 

like 

unfair 

fair 

sufficient 

* 

• 

• 

insufficient 

useless 

• 

• 

• 

useful 

adequate 

• 

inadequate 

valuable 

worthless 

irrelevant 

relevant 

approve 

* 

disapprove 

unimportant 

Important 

Comments:  Please 

comment  on  what 

it  is 

that  you 

like 

best 

about  the 

concept 

you  have  rated 

above 

and  what 

it  is 

you 

like  least. 

Like  best: 


Like  least: 

I 

1 


Other  oomments  (use  other  side  if  necessary): 
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t ' 

V! 


•VI 


1; 

H 


CMI  TEST  FEEDBACK  TIME 


effective 

: 

ineffective 

fast 

• 

• 

slow 

bad 

• 

* 

• 

• 

good 

unpredictable 

predictable 

regular 

irregular 

consistent 

inconsistent 

inefficient 

efficient 

reliable 

unreliable 

insufficient 

sufficient 

inadequate 

adequate 

Comments; 


Please  comment  on  what  it  is  that  you  like  best  about  the 
concept  you  have  rated  above  and  what  it  is  you  like  least. 


Like  best; 


Like  least; 


Other  comments  (use  other  side  if  necessary) ; 
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Appendix  I,  Exhibit  2 

THE  SEMANTIC  DIFFERENTIAL  SURVEY  INSTRUMENT 
FOR  KEY  SUPPORT  PERSONNEL 


DIRECTIONS;  On  the  following  pages  we  are  asking  you  to  rate  your  feelings 
toward  a number  of  concepts/things  related  to  your  training.  In  accordance 
with  SECNAVINST  5211.5  of  14  Aug  1975,  information  included  in  the  CMI  survey 
is  requested  under  the  authority  of  Departmental  Regula- 

tions. This  information  is  required  to  assist  in  the  evaluation  of  the  CMI 
System.  You  are  not  required  to  provide  this  information,  but  you  can  be  as- 
sured that  your  individual  answers  will  be  kept  completely  confidential  and  no 
attempt  will  be  made  to  evaluate  you  or  the  command  you  are  currently  with 
based  on  your  answers. 

On  the  following  pages  you  are  asked  to  rate  your  feelings  on  the  following 
concepts;  COURSE  MATERIAL,  COMPUTER  MANAGED  INSTRUCTION,  LEARNING  SUPERVISOR, 
STUDY  CENTER,  CMI  TESTING  PROCEDURES,  CONTENT  OF  CMI  TEST  FEEDBACK,  CMI  TEST 
FEEDBACK  TIME.  (All  but  the  STUDENT  questions  are  the  same  as  the  forms  for 
the  student  survey  instrument.  Therefore  only  the  STUDENT  question  sheet  is 
included  in  this  exhibit.) 

Here  is  how  to  use  the  scales; 


If  you  feel  the  scale  relates  very  closely  to  your  feelings  mark  it  as  such; 


strong 

X 

; 

: : 

; 

» • 

• • 

weak 

strong 

or 

• • 

• • 

X 

weak 

If  you 

feel 

the  scale 

relates  quite  closely 

to 

your 

feelings  mark  it 

as 

such; 

strong 

; X 

• • 

• 

; . 

weak 

strong 

or 

; X ; 

weak 

If  you 

feel 

the  scale 

relates  only  slightly 

to 

your 

feelings  mark  it 

as 

such; 

strong 

• 

; X ; 

* 

• « 

weak 

strong 

• 

or 

• • 

• • 

• 

X 

• • 

• • 

weak 
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If  you  feel  you  are  neutral  or  feel  the  scale  is  not  relevant  to  your  feel- 
ings, mark  it  as  sucTTl 

strong  ; ; : X : : s weak 


PLEASE  REMEMBER  THE  FOLLOWING: 


1.  Mark  all  scales;  do  not  skip  any. 

2.  Place  your  mark  in  the  middle  of  each  line,  not  in  between  lines. 

3.  Work  quickly  and  do  not  put  more  than  one  mark  for  each  scale. 

4.  Please  be  sure  that  you  have  filled  in  the  necessary  information  on  the 
cover  page  of  the  survey. 
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THE  STUDENTS  YOU  HAVE  WORKED  WITH 
DURING  THIS  TRAINING  PERIOD 


qualified 

stupid 


skilled 


foolish 


unreliable 


adequate 


ineffective 


dislike 


superior 


unqualified 

intelligent 


unskilled 


reliable 


inadequate 


effective 


inferior 


Conments:  Please  coiranent  on  what  it  is  that  you  like  best  about  the 
concept  you  have  rated  above  and  what  it  is  you  like  least. 


Like  best: 


Like  least: 


Other  oomnente  (use  other  side  if  necessary) : 
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Appendix  I,  EXHIBIT  3 
POST  INTERWIEW  QUESTIONS  FOR 
STUDENTS  AND  KEY  PERSONNEL 


1.  Oil  Concept  Questions. 


Generally,  how  do  you  feel  about  the  idea  of  computer  managd  instruction 
(CMD? 


How  would  you  compare  CMI  to  other  forms  of  Naval  training  you  have  had 
in  the  past? 


Why  did  you  decide  to  take  this  course? 


Have  you  ever  taken  any  other  computer  managed  instruction  type  course? 
If  so,  what  was  it? 


If  the  opportunity  presented  itself,  would  you  take  another  CMI  course? 


2.  Study  Center  Questions. 


Generally  what  are  your  feelings  about  this  training  center  (schoolhouse) ? 


Were  there  any  problems  in  the  study  center  that  you  feel  are  serious 
enough  to  be  mentioned  now? 
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Testinq/Study  Procedure  Questions. 

Generally,  what  is  your  feeling  about  the  testing  procedure  for  this 
course? 

Generally,  how  do  you  feel  about  the  feec3back  you  received? 

Generally,  how  do  you  feel  about  the  feedback  time  interval  — 
the  time  between  taking  a test  and  getting  back  your  test  results  — 
during  the  course? 

Generally,  how  do  you  feel  about  the  time  interval  between  study  sessions 
during  the  course? 

+ 


Learning  Supervisor  Questions. 

Generally,  how  do  you  feel  about  the  learning  supervisor  for  this  course? 

How  do  you  feel  about  the  help  you  received  from  your  learning  supervisor 
during  the  course? 

Did  you  feel  comfortable  when  consulting  with  your  learning  supervisor 
during  the  course? 

+ 


The  CMI  Course  and  Course  Material  Questions. 

What  are  some  of  the  things  you  like  best  about  the  course  and  training 
you  have  received  here? 

What  are  some  of  the  things  you  like  least  about  the  course  and  training 
you  have  received  here? 
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Was  the  training  you  received  adequate?  Did  the  course  meet  your  needs? 
(If  not,  why?) 

How  was  your  motivation  during  the  course? 

How  do  you  feel  about  the  instructional  materials  that  were  used  for  the 
course  (PI  books,  etc.)? 

How  did  you  find  the  instructional  materials  in  terms  of  your  reading  and 
understanding? 

How  do  you  feel  about  the  practical  'hands  on'  materials  in  the  course? 


6.  Operational/Procedural  Questions. 

Generally,  how  do  you  feel  about  the  idea  of  taking  technical  training  at 
an  operational  site  as  compared  with  taking  technical  training  at  a 
designated  technical  school? 

If  given  the  choice,  which  would  you  prefer: 

• Training  at  your  tour  station? 

• Or  training  at  a special  technical  school? 

How  did  you  find  this  course  in  relation  to  your  regular  assigned  work 
duties? 

Did  you  run  into  any  problems  while  taking  the  training  which  you  feel 
are  serious  enough  to  be  mentioned? 
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Generally,  how  do  you  feel  about  taking  a technical  course  in  preparation 
for  your  rate  advancement  exam? 

All  in  all,  how  did  the  training  you  received  here  affect  you  in  terms  of 
your  job?  Your  career  advancement?  Any  other  ways  that  you  can  think  of? 

+ 


General  Demographic  Questions. 

How  long  have  you  been  in  the  Navy? 

How  long  have  you  been  at  this  site? 

How  long  have  you  been  at  this  paygrade/rank? 

Describe  what  you  do  on  your  job. 

Before  your  present  position/job  what  did  you  do? 

What  other  kinds  of  technical  training  have  you  had  at  particular 
schools — training  centers? 
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STUDENT,  KEY  PERSONNEL  AND  MAINTENANCE  LOGS  ;! 


This  appendix  contains  samples  of  the  following  forms: 

• CMI  Student  Time  Record 

• Summary  of  Correspondence 
Student  Progress 

e Key  Personnel  Time  Log 

• Maintenance  Log 


I 


> 


> 
j ' 
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Appendix  K 

SUPPORT  REQUIREMENTS--INTERVIEW  QUESTIONS 

As  discussed  in  the  chapter  on  demonstration  design,  key  site  per- 
sonnel and  students  will  be  interviewed  to  assess  the  support  require- 
ments for  operational  CMI  systems.  The  requirements  of  concern  are; 

e Personnel 

• Personnel  training 

• Organization  and  management  structure 

• Space  requirements  and  operational  procedures 

• Equipment,  maintenance,  spare  parts  and  logistics  requirements 

These  correspond  to  five  of  the  research  objectives  of  the  demon- 
stration. Based  on  the  Issues  involved  in  determining  these  require- 
ments, interview  questions  have  been  developed  for  each  of  the 
following: 

• Commanding  Officer,  Communications  Officer,  Division  Offi- 
cers, watch  supervisors,  chiefs  of  the  watch,  and  leading 
chiefs 

• ESO,  DTC/POs,  LS,  and  CMI  advisor 

• ETs 

• Students 

The  Interviews  will  be  given  at  the  end  of  the  first  two  weeks  of 
the  demonstration,  at  the  end  of  the  first  month,  and  once  a month  af- 
ter that  until  the  end  of  the  demonstration. 
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INTERVIEW  QUESTIONS:  Conunanding  Officer  NAVCOMMSTA  Stockton 

Conmiunlcatlons  Officer 
Division  Officers 
Watch  Supervisors 
Chiefs  of  the  watch 
Leading  chiefs 

1.  Are  you  satisfied  with  the  progress  to  date  of  the  COMISAT  demon- 
stration? 

2.  Do  you  feel  that  you  have  been  adequately  informed  regarding  the 
status  of  the  demonstration?  If  not,  what  change  in  reporting  is 
desirable? 

3.  Have  you  been  properly  informed  of  problems  with  the  demonstration? 

4.  Do  you  feel  the  additional  workload  imposed  on  base  personnel  by 
the  demonstration  is  acceptable?  What  changes  would  you  recommend? 

5.  Have  communication  operations  been  affected  by  the  existence  of 
the  demonstration?  If  so,  how?  What  changes  would  you  recommend 
for  the  CMI  demonstration  to  avoid  a negative  effect  on  operations? 

6.  Are  you  satisfied  with  the  manner  in  which  your  direction  or  sug- 
gestions regarding  demonstration  activities  have  been  carried 
out?  Are  there  any  issues  of  communication  problems  or  divided 
authority  that  should  be  resolved? 

7.  Are  there  any  other  problems  you  would  like  to  discuss?  Do  you 
have  any  suggestions  for  dealing  with  them? 
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INTERVIEW  QUESTIONS! 


1. 


6. 


ESO 

DTC/POs 

LS 

CMI  Ad;/isor 


Are  you  satisfied  with  the  opportunity  you  have  to  apprise  the 
proper  authority  of  problems  or  ideas  for  improving  the  demonstra- 
tion? Are  you  satisfied  with  the  time  it  takes  for  decisions  to 
be  made? 


2.  Do  you  feel  demonstration  matters  are  given  enough  attention  by 
students  and  staff? 


When  there  are  changes  in  the  demonstration  or  in  your  regular 
duties,  are  you  given  enough  time  to  prepare  for  an  orderly 
transition? 


4.  Are  you  satisfied  with  your  participation  in  decisions  concerning 
the  demonstration? 


Do  you  have  enough  time  to  carry  out  your  role  in  the  demonstra- 
tion? If  not,  what  changes  in  staffing  or  procedure  would  you 
suggest? 


Are  you  satisfied  with  the  following  aspects  of  the  demonstration 
facilities  and  operating  procedures? 


Learning  center  space  and  facilities,  especially  the  use  of 
the  conference  room  tables  for  study,  lab  work  and  exams 


The  scheduling  of  study,  lab  work  and  exams.  Including  use  of 
materials 


The  scheduling  of  student  Interaction  with  the  LS  and  ETs 


r » 


h! 


} ■ 


I ^ 

V* 

1 


i. 


TT 
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(Question  6 continued) 


• The  procedures  for  keeping  student  and  key  personnel  logs  and 
entering  study  time  into  the  computer 


7.  Are  there  any  problems?  Do  you  have  suggestions  for  resolving 
them? 
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INTERVIEW  QUESTIONS;  Electronic  Technicians 

1.  Are  you  satisfied  with  the  opportunity  you  have  to  apprise  the 
proper  authority  of  problems  or  ideas  for  improving  the 
demonstration?  Are  you  satisfied  with  the  time  it  takes  for 
decisions  to  be  made? 

2.  Do  you  feel  demonstration  matters  are  given  enough  attention  by 
your  superiors  and  other  staff? 

3.  When  there  are  changes  in  the  demonstration  or  in  your  regular 
duties,  are  you  given  enough  time  to  prepare  for  an  orderly 
transition? 

4.  Are  you  satisfied  with  your  participation  in  decisions  concerning 
the  demonstration? 

5.  Have  you  been  satisfactorily  prepared  to  deal  with  the  questions 
raised  by  the  students  on  technical  matters?  On  administrative 
matters? 

6.  When  you  have  to  substitute  for  the  learning  supervisor,  do  you 
consider  your  performance  of  normal  duties  to  be  seriously  af- 
fected. If  so,  what  specific  functions  are  most  affected,  and 
how?  What  changes  would  you  recommend  to  correct  the  situation? 

7.  Do  you  find  that  maintenance  work  in  support  of  the  demonstration 
reduces  your  performance  of  regular  duties?  If  so,  what  functions 
are  affected,  and  how?  Would  you  recommend  changes  in  procedures 
to  overcome  the  problem? 
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8.  Are  you  satisfied  with  the  following  aspects  of  tne  demonstration 
facilities  and  operating  procedures? 

e Learning  center  space  and  facilities,  especially  the  use  of 
the  conference  room  tables  for  study,  lab  work  and  exams 

e The  scheduling  of  study,  lab  work  and  exams,  including  use  of 
materials 

e The  scheduling  of  their  interaction  with  the  students 
e The  tasks  and  their  scheduling  for  ETs 

e The  procedures  for  keeping  student  and  key  personnel  logs  and 

entering  study  time  into  the  computer 

9.  Are  there  any  problems?  Do  you  have  suggestions  for  resolving 
them? 
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INTERVIEW  QUESTIONS:  Students 

1.  Are  you  satisfied  with  the  opportunity  you  have  to  apprise  the 
proper  authority  of  problems  or  ideas  for  improving  the  demonstra- 
tion? Are  decisions  reached  in  a timely  manner? 

2.  Do  you  feel  demonstration  matters  are  given  enough  attention  by 
your  superiors  and  other  staff? 

3.  When  there  ate  changes  in  the  demonstration  or  in  your  regular 
duties,  are  you  given  enough  time  to  prepare  for  an  orderly 
transition? 

4.  Do  you  consider  your  performance  of  normal  duties  to  be  satisfac- 
tory under  the  revised  schedule  set  for  the  demonstration?  If 
not,  what  functions  are  most  affected,  and  how?  What  changes 
would  you  recommend  to  resolve  the  problem? 

5.  Are  you  satisfied  with  the  following  aspects  of  the  demonstration 
facilities  and  operating  procedures? 

• Learning  center  space  and  facilities,  especially  the  use  of 
the  conference  room  tables  for  study,  lab  work  and  exams 

• The  scheduling  of  study,  lab  work  and  exeuns,  including  use  of 
materials 

• The  scheduling  of  interaction  with  the  LS  and  ETs 

• The  tasks  and  their  scheduling  for  ETs 

• The  procedures  for  keeping  student  time  records,  logs,  and 

entering  study  time  into  the  computer 

6.  What  other  problems  exist?  Do  you  have  suggestions  for  resolving 
them? 
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Appendix  L 

RECORDKEEPING  AND  PROGRESS  REPORTING  MATERIALS 

This  appendix  contains  materials  related  to  recordkeeping  and  progress 
reporting  for  the  CMI  course.  The  appendix  contains: 

• A Laboratory  and  Performance  Progress  Sheet 

• A Student  Progress  Sheet  for  LS  use 

• A Study/Progress  Sheet  for  student  use 
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STUDENT  STUDY/PROGRESS 
SHEET 

STUDY  PROCEDURES 


This  procedure  sheet  is  furnished  for  your  personal  use.  To  progress  through 
this  course  in  the  proper  sequence,  you  should  do  each  of  the  items  on  this 
sheet  in  the  same  order  in  which  they  are  listed.  You  may  use  the  blank  space 
by  each  item  to  check  off  the  item  or  write  in  the  work  day  when  you  have 
completed  each  item.  If  you  have  any  questions  about  this  procedure  sheet, 
contact  your  Learning  Supervisor. 

ITEM  WORK  DAY 

1.  Take  pre-math  test.  

2.  Take  math  Test  One  (if  applicable) . 

3.  Study  Module  One,  all  lessons;  take  Mod  1 test.  

4.  Study  Module  Two,  all  lessons;  take  Mod  2 test.  

5.  Study  Module  Three,  all  lessons;  do  lab  experiment  3-4.  

6.  Take  Performance  Test  3.  

7.  Review  Module  Three,  all  lessons;  take  Mod  3 test  

8.  Study  Module  Four,  Lesson  One;  do  lab  experiment  4-1.  
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9.  Study  Module  Four,  Lessons  Two  and  Three;  do  lab  experiment 

4- 3. 

10.  Take  Performance  Test  4. 

11.  Review  Module  Four,  all  lessons;  take  Mod  4 test. 

12.  Taka  math  Test  Two  (if  applicable). 

13.  Study  Module  Five,  Lessons  One  and  Two;  do  lab  experiment 

5- 2. 

14.  Review  Module  Five,  Lessons  One  and  Two;  take  Mod  5-1  test. 

15.  Study  Module  Five,  Lessons  Three  thru  Five;  do  lab 
experiment  5-5. 

16.  Take  Performance  Test  5. 

17.  Review  Module  Five,  Lessons  Three  thru  Five;  take  Mod  5-2 
test. 

18.  Take  math  Test  Three  (if  applicable). 

19.  Study  Module  Six,  Lessons  One  and  Two;  do  lab  experiment 


20.  Study  Module  Six,  Lessons  Three  and  Four;  do  lab  experiment 


21.  Take  Performance  Test  6. 


22.  Review  Module  Six,  all  lessons;  take  Mod  6 test. 


TAEG  Report  No.  49 


• J 


23.  Study  Module  Seven,  Lesson  One;  do  lab  experiment  7-1. 

24.  Take  Performance  Test  7. 

25.  Review  Module  Seven,  Lesson  One;  take  Mod  7-1  test. 

26.  Study  Module  Seven,  Lessons  Two  and  Three;  take  Mod  7-2  test. 

27.  Review  Modules  One  thru  Seven;  take  the  DC  comprehensive 
examination. 

28.  Study  Module  Eight,  all  lessons;  take  Mod  8 test. 

29.  Take  math  Test  Four  (if  applicable). 

30.  Study  Module  Nine,  all  lessons;  do  lab  experiment  9-6. 

31.  Review  Module  Nine,  all  lessons;  take  Mod  9 test. 

32.  Take  math  Test  Five  (if  applicable). 

33.  Study  Module  Ten,  Lessons  One  thru  Five;  do  lab  experiment 
10-5. 


34.  Study  Module  Ten,  Lesson  Six;  review  Lessons  One  thru  Five; 
take  Mod  10  test. 

35.  Study  Module  Eleven,  Lessons  One  thru  Four;  do  lab 
experiment  11. 

36.  Review  Module  Eleven,  Lessons  One  thru  Four;  take  Mod  11-1 
test. 
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37.  Study  Module  Eleven,  Lessons  Five  thru  Seven;  take  Mod  11-1 
test. 

38.  Take  math  Test  Six  (if  applicable). 

39.  Study  Module  Twelve,  Lessons  One  and  Two;  take  Mod  12-1 
test. 

40.  Study  Module  Twelve,  Lesson  Three;  take  Mod  12-1  test. 

41.  Study  Module  Twelve,  Lessons  Four  thru  Six;  take  Mod  12-3 
test. 

42.  Study  Module  Thirteen,  all  lessons;  view  sound  slide 
presentation  13-4  (WV-77-E) ; do  lab  experiment  13. 

43.  Take  Performance  Test  13. 

44.  Review  Module  Thirteen,  all  lessons;  take  Mod  13  test. 

45.  Study  Module  Fourteen,  Lessons  One  thru  Three;  take  Mod 
14-1  test. 

46.  Study  Module  Fourteen,  Lessons  Four  and  Five;  do  lab 
experiment  14. 

47.  Review  Module  Fourteen,  Lessons  Pour  and  Five;  take  Mod 
14-2  test. 

46.  Review  Modules  Eight  thru  Fourteen;  take  the  AC 
comprehensive  examination. 
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Appendix  M 

ANALYSIS  OF  COMMUNICATIONS  LINE  OPTIONS 


1 . Alternatives  Considered 

Three  land  line  alternatives  were  considered: 

• Alternative  1:  A dedicated  phone  line  leased  from  Pacific 
Telephone  connecting  the  Stockton  cluster  directly  with  the 
San  Diego  concentrator  at  Building  94.  This  option  requires 
the  use  of  a 202-T  modem  at  each  end  of  the  line.  Since  this 
line  would  always  be  available,  CMI  test  sheets  could  be  in- 
serted at  any  time  that  the  San  Diego  concentrator  and 
Memphis  CMI  computer  were  operating.  The  Memphis  computer 
operates  from  0500  to  2030  CDT.  The  San  Diego  concentrator 
operates  from  0600  PDT  (0800  CDT)  to  1830  PDT  (2030  CDT). 

• Alternative  2;  A commercial  dial  line  connecting  the 
Stockton  cluster  to  the  MIISA  San  Diego  concentrator  when  C3il 
messages  are  to  be  sent.  This  alternative  requires  a 202-S 
data  set  and  a duplex  arrangement  at  each  end  of  the  line. 

• Alternative  3:  A hybrid  communications  system  which  uses  one 
of  the  two  3KHZ,  4-wire  phone  lines  currently  installed  be- 
tween Stockton  Tech  Control  and  NAVCOMMSTA  San  Diego  Tech 
control  when  not  required  for  Navy  operations.  These  lines 
are  available  to  permit  Stockton  to  assume  some  of  the  East 
Pac  full  period  termination  requirements.  While  the  past 
tactical  usage  of  these  lines  has  been  low,  it  is  anticipated 
that  future  usage  will  be  considerably  increased  and  hence 
the  amount  of  time  they  would  be  available  to  the  CMI  demon- 
stration is  uncertain.  The  back-up  system  would  be  the  com- 
mercial dial  up  system  (Alternative  2) . Because  the  dedi- 
cated and  dial  services  require  different  modems,  two  systems 
are  needed: 

The  primary  system  would  consist  of  a 202-T  modem 
connected  to  a phone  line  to  be  installed  from  the 
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Stockton  cluster  to  Stockton  Tech  Control.  From  there  it 
would  be  connected  to  one  of  the  tactical  lines  to 
San  Diego  Tech  Control.  From  here  a commercial  phone  line 
would  connect  to  the  San  Diego  concentrator  through  another 
202-T  modem. 

— All  of  the  equipment  of  Alternative  2 would  still  be 
required  as  a full-time  backup  system.  Thus,  the 
initial  installation  costs  would  be  greater  than 
Alternative  1 or  2.  The  only  savings  would  be  in 
operating  costs. 


In  all  three  cases  an  RP6352  asynchronous  line  modulator  is  needed 
to  connect  the  line  to  the  'CMI  concentrator. 


2.  Comparison 

Table  M-1  shows  the  cost  differences  between  the  alternatives. 
The  totals  are  as  follows: 


Alternative  1,  Commercial  Dedicated 


$2,602 


Alternative  2,  Commercial  Dial 


$2,321  plus  software 
modifications 


Alternative  3,  Navy  and  Dial  Backup  $2,200  plus  software 

modifications 


f ' 
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Table  M- 1.  Cost  of  Communication  Line  Options  for  Six  Months 


Alternative  1;  Commercial  Dedicated 


Component 


Cost 


Stockton: 

• 202-T  dedicated  line  modem 

Stockton  - San  Diego 

• Dedicated  commercial  line 

San  Diego 

• NTC  202-T  dedicated  line  modems 

• Asynchronous  line  modulator  RP6352 

Total 


$ 150  ($25/mo) 

55  (installation) 


1,800  ($300/mo) 
200  (installation) 


150  ($25/mo) 

55  (installation) 

192  ($32/ mo) 


$2,602 
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Alternative  2;  Commercial  Dial 


Component 


Stockton: 

• 202-S  data  set,  duplex  arrangement  $ 270 

75 

87 


Cost 


(45/mo  202-SI 
(installation) 

($14. 50/mo  duplex) 


Stockton  • San  Diego 

• Two  one-pair  commercial  lines 

• Toll  charges  (est.) 


175  ($29.10/mo) 
90  (installation) 
1,000 


San  Diego:' 

• 202-S  data  set,  duplex  arrangement  270  ($45/ mo  202-S) 

75  (installation) 

87  ($14. 50/mo  duplex) 

• Asynchronous  line  modulator  RP6352  192  ($32/ mo) 

Total  $2,321 


'"Exlarwv*"  software  changes  to  the  concentrator  at  San  Diego  believed  to  be  in  the  order  of  fifteen 
hundred  dollart  would  be  required  to  accommodate  a dial  line. 
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Table  M-f.  {Continued) 


Alternative  3:  Navy  Line  With  Dial  Backup^ 

Component 

Cost 

Stockton: 

• Connection  to  Tech  Control 

$ 400 

• Backup  line  202-S  data  set  duplex 

270  ($45/mo 202-S) 

arrangement 

75  (installation) 

• 202-T  dedicated  line  modem 

87  ($14. 50/mo.  duplex) 
150  ($25/mo) 

55  (installation) 

Stockton  - San  Diego 

• Navy  line 

0 

• Two  one-pair  commercial  lines 

175  ($29. 10/mo) 

90  (installation) 

San  Diego 

• NCS-NTC  line 

84  ($14/mo) 

40  (installation) 

• 202-S  data  set,  duplex 

270  ($45/mo 202-S) 

arrangement  for  backup  dial  line 

75  (installation) 

87  ($14.50/mo  duplex) 

• 202-T  dedicated  line  modem 

150  ($25/mo) 

• Asynchronous  line  modulator  RP6352 

192  ($32/moeach) 

Total 

$2,200 

'A  portion  of  th«  dial  charges  shown  In  the  commercial  dial  option  would  have  to  be  added  Also,  the 
same  software  changes  to  the  concentrator  as  Alternative  1 of  at  laast  fifteen  hundred  dollars  would  be 
required  to  accommodate  a dial  backup. 
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Appendix  N 

SAMPLE  TASKING  LETTER 


SPEEDLETTER 


Prom:  Chief  of  Naval  Education  and  Training 
To:  Chief  of  Naval  Technical  Training 

Commander,  Naval  Telecommunications  Command 

CO,  Management  Instructional  Information  Systems  Activity 

Subj:  Phase  III  COMISAT  Project  Requirements;  tasking  of 

Ref:  (a)  Contract  No. 

1.  In  order  to  meet  the  demonstration  requirements  of  the  COMISAT 
project  (reference  (a)),  certain  timely  actions  are  required.  To  date, 
two  phases  have  been  completed.  Phase  I,  the  feasibility  study,  deter- 
mined that  distribution  of  CMI  to  ships  and  land  bases  was  possible, 
although  certain  constraints  were  identified.  Phase  II  developed  a 
practical  design  for  use  within  a Navy  setting.  Phase  III  involves  the 
preparation  for  the  demonstration.  Two  other  phases  will  follow: 

Phase  IV,  the  demonstration,  and  Phase  V,  the  evaluation. 

2.  The  following  requested  actions  are  required  in  preparation  for 
the  demonstration.  Direct  liaison  with  action  and  information  addres- 
sees is  authorized  in  order  to  determine  detailed  specifications. 

a.  CNTBCHTRA 

(1)  Provide  ten  (10)  complete  sets  of  BB/E  Course  69  texts. 
Modules  1-14,  and  test  sheets,  lab  papers,  Opscan  administrative  forms 
(PI,  P2,  P3)  and  terminal  paper  sufficient  to  support  the  experiment. 

To  be  onsite  by  20  August  1977.  Coordinate  with  the  contractor. 

(2)  Provide  two  (2)  complete  sets  of  all  support  equipment 
for  Course  BB/E  69,  Modules  1-14,  to  be  onsite  by  20  August  1977. 

(3)  Provide  t«ro  (2)  full  sets  of  NEAT  testing  bounds  on  a no 
cost  loan,  modified  with  a box  covering  the  test  control  knobs.  Also, 
provide  other  related  oourse/lab  equipment  as  specified  by  PRC.  Equip- 
ments to  be  onsite  by  20  August  1977. 


N-1 


TAEG  Report  No.  49 


(4)  Train  and  assign  a Learning  Center  Supervisor.  Assigned 
individual  to  be  on  station  at  NAVCOMMSTA,  Stockton,  California  by  20 
August  1977.  TAD/TVL  funds  to  be  provided  by  CNET.  IT-BE/E  training 
should  start  by  18  July  1977.  In  order  to  preclude  personnel  or  com- 
mand inconveniences,  more  than  one  instructor  may  be  used  as  a replace- 
ment during  the  duration  of  the  demonstration. 

(5)  Coordinate  with  MIISA  in  identifying  and  monitoring 
files  on  control  and  study  groups  for  collecting  data  on  performance 
and  attitudes.  This  will  require  a search  of  Course  69  records  for  a 
historic  performance  control  group.  Appropriate  functional  description 
to  be  provided  to  MIISA  by  13  July  1977,  by  the  contractor. 

(6)  BE/E  TPC  assist  in  identifying  a group  of  55  trainees  to 
validate  the  attitude  test  at  Great  Lakes  Training  Center,  and  55 
trainees  at  Memphis.  Validation  to  commence  18  July  1977.  Validation 
to  be  completed  by  12  August  1977.  Specifications  to  be  provided  by 
the  contractor. 

(7)  Assign  a staff  member  to  oversee  the  setup  of  the  opera- 
ting equipment  (Training  and  Oil)  at  NAVCOMMSTA,  Stockton,  California. 
Limited  short  term  TAD/TVL  funding  to  be  provided  by  CNET.  Coordinate 
with  CNET  prior  to  commitment  of  funds. 

b.  MIISA 

(\)  Provide  NAVCOMMSTA,  Stockton,  California  with  two  (2) 
operative  OPSCAN/Terminet  clusters  by  29  August  1977.  Parallel  acqui- 
sition paths  should  be  concurrently  pursued  in  order  to  assure  selec- 
tion of  the  earliest  single  delivery  source.  This  may  include  modifi- 
cation of  the  existing  Honeywell/OPSCAN  Contract  and  investigation  of 
GSA  Requirements  Contracts. 

(2)  Provide  standard  operative  CMI  telecommunication  cir- 
cuits and  associated  equipments  to  support  the  CMI  equipment  in  Stock- 
ton.  Operational  cutover  to  coincide  with  OPSCAN/Terminet  requirements. 

(3)  Request  MIISA  DET,  San  Diego,  to  prepare  and  mail  in  two 
(2)  day  intervals  during  the  first  month  of  the  demonstration  and 
weekly  thereafter,  student  reports  and  other  administrative  support  re- 
ports. Also,  MIISA  DET,  San  Diego,  is  to  coordinate  change  and  error 
resolution  with  the  Stockton  Learning  Center  Supervisor. 

(4)  Request  MIISA  DET,  Memphis,  to  establish  a Complex  Iden- 
tification Number  for  the  Stockton  trainees,  estimate  time  to  comple- 
tion for  a group  of  trainees  who  are  representative  of  Stockton  based 
on  data  supplied  by  the  contractor. 
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(5)  All  technical  specifications  are  available  from  the  con- 
tractor. HIISA  funding  requirements  not  presently  allowed  for  are  to 
be  provided  for  by  CNET.  Prior  coordination  is  required  before  commit- 
ment of  funds. 

(6)  Limited  short  term  TAD/IVL  funding  to  be  provided  by 
CNET,  for  MIISA  DET  personnel.  Coordinate  prior  to  commitment  of  funds. 

C.  COMNAVTErXX?M 

(1)  Commence  preliminary  preparation  leading  to  the  receipt 
of  detailed  specifications  from  MIISA  for  a leased  telecommunications 
line  connecting  the  NAVOOMISTA,  Stockton  learning  complex  with  MIISA 
DET,  San  Diego.  Technical  data  is  available  from  the  contractor.  The 
line  should  be  activated  by  29  August  1977.  All  leased  item  funds  are 
to  be  provided  by  CNET.  Prior  coordination  is  required  before  commit- 
ment of  funds. 

3.  In  order  to  maintain  an  open  line  of  communications  the  following 
points  of  contact  are  established  for  coordination  purposes; 

CNET 

CNTECHTRA 

Contractor 

TAEG 

DARPA 

MIISA  DET,  Memphis 
MIISA  DET,  San  Diego 
COMNAVTELOOM 
NAVCOMMSTA,  Stockton,  CA 

4.  Submit  all  costs  and  time  charged  to  the  project  in  a form  suit- 
able for  analysis  as  specified  by  the  contractor.  The  time  frame  for 
the  demonstration  phase  of  the  project  is  from  September  1977  to  March 
1978,  allowing  for  setup  and  takedown  time. 

5.  Detail  specifications  for  above  action  items  are  available  from 
the  contractor.  Provide  telephonic  verification  of  firm  delivery 
dates,  and  completion  of  action  items  set  forth  to  the  Contracting  Of- 
ficer's Technical  Representative  (OOTR) . Notify  the  COTR  in  the  event 
of  delivery/oompletlon/schedule  problems. 
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